tilt-shift-camera/app/src/main/java/no/naiv/tiltshift/effect/TiltShiftShader.kt

157 lines
6.2 KiB
Kotlin
Raw Normal View History

package no.naiv.tiltshift.effect
import android.content.Context
import android.opengl.GLES11Ext
import android.opengl.GLES20
import no.naiv.tiltshift.R
Fix concurrency, lifecycle, performance, and config issues from audit Concurrency & bitmap lifecycle: - Defer bitmap recycling by one cycle so Compose finishes drawing before native memory is freed (preview bitmaps, thumbnails) - Make galleryPreviewSource @Volatile for cross-thread visibility - Join preview job before recycling source bitmap in cancelGalleryPreview() to prevent use-after-free during CPU blur loop - Add @Volatile to TiltShiftRenderer.currentTexCoords (UI/GL thread race) - Fix error dismiss race with cancellable Job tracking Lifecycle & resource management: - Release GL resources via glSurfaceView.queueEvent (must run on GL thread) - Pause GLSurfaceView when entering gallery preview mode - Shut down captureExecutor in CameraManager.release() (thread leak) - Use WeakReference for lifecycleOwnerRef to avoid Activity GC delay - Fix thumbnail bitmap leak on coroutine cancellation (add to finally) - Guarantee imageProxy.close() in finally block Performance: - Compute gradient mask at 1/4 resolution with bilinear upscale (~93% less per-pixel trig work, ~75% less mask memory) - Precompute cos/sin on CPU, pass as uCosAngle/uSinAngle uniforms (eliminates per-fragment transcendental calls in GLSL) - Unroll 9-tap Gaussian blur kernel (avoids integer-branched weight lookup that de-optimizes on mobile GPUs) - Add 80ms debounce to preview recomputation during slider drags Silent failure fixes: - Check bitmap.compress() return value; report error on failure - Log all loadBitmapFromUri null paths (stream, dimensions, decode) - Surface preview computation errors and ActivityNotFoundException to user - Return boolean from writeExifToUri, log at ERROR level - Wrap gallery preview downscale in try-catch (OOM protection) Config: - Add ACCESS_MEDIA_LOCATION permission (GPS EXIF on Android 10+) - Accept coarse-only location grant for geotags - Remove dead adjustResize (no effect with edge-to-edge) - Set windowBackground to black (eliminates white flash on cold start) - Add values-night theme for dark mode - Remove overly broad ProGuard keeps (CameraX/GMS ship consumer rules) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:44:12 +01:00
import kotlin.math.cos
import kotlin.math.sin
import java.io.BufferedReader
import java.io.InputStreamReader
/**
* Manages OpenGL shader programs for the tilt-shift effect.
*/
class TiltShiftShader(private val context: Context) {
var programId: Int = 0
private set
// Attribute locations
var aPositionLocation: Int = 0
private set
var aTexCoordLocation: Int = 0
private set
// Uniform locations
private var uTextureLocation: Int = 0
private var uModeLocation: Int = 0
private var uIsFrontCameraLocation: Int = 0
private var uAngleLocation: Int = 0
private var uPositionXLocation: Int = 0
private var uPositionYLocation: Int = 0
private var uSizeLocation: Int = 0
private var uBlurAmountLocation: Int = 0
private var uFalloffLocation: Int = 0
private var uAspectRatioLocation: Int = 0
private var uResolutionLocation: Int = 0
Fix concurrency, lifecycle, performance, and config issues from audit Concurrency & bitmap lifecycle: - Defer bitmap recycling by one cycle so Compose finishes drawing before native memory is freed (preview bitmaps, thumbnails) - Make galleryPreviewSource @Volatile for cross-thread visibility - Join preview job before recycling source bitmap in cancelGalleryPreview() to prevent use-after-free during CPU blur loop - Add @Volatile to TiltShiftRenderer.currentTexCoords (UI/GL thread race) - Fix error dismiss race with cancellable Job tracking Lifecycle & resource management: - Release GL resources via glSurfaceView.queueEvent (must run on GL thread) - Pause GLSurfaceView when entering gallery preview mode - Shut down captureExecutor in CameraManager.release() (thread leak) - Use WeakReference for lifecycleOwnerRef to avoid Activity GC delay - Fix thumbnail bitmap leak on coroutine cancellation (add to finally) - Guarantee imageProxy.close() in finally block Performance: - Compute gradient mask at 1/4 resolution with bilinear upscale (~93% less per-pixel trig work, ~75% less mask memory) - Precompute cos/sin on CPU, pass as uCosAngle/uSinAngle uniforms (eliminates per-fragment transcendental calls in GLSL) - Unroll 9-tap Gaussian blur kernel (avoids integer-branched weight lookup that de-optimizes on mobile GPUs) - Add 80ms debounce to preview recomputation during slider drags Silent failure fixes: - Check bitmap.compress() return value; report error on failure - Log all loadBitmapFromUri null paths (stream, dimensions, decode) - Surface preview computation errors and ActivityNotFoundException to user - Return boolean from writeExifToUri, log at ERROR level - Wrap gallery preview downscale in try-catch (OOM protection) Config: - Add ACCESS_MEDIA_LOCATION permission (GPS EXIF on Android 10+) - Accept coarse-only location grant for geotags - Remove dead adjustResize (no effect with edge-to-edge) - Set windowBackground to black (eliminates white flash on cold start) - Add values-night theme for dark mode - Remove overly broad ProGuard keeps (CameraX/GMS ship consumer rules) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:44:12 +01:00
private var uCosAngleLocation: Int = 0
private var uSinAngleLocation: Int = 0
/**
* Compiles and links the shader program.
* Must be called from GL thread.
*/
fun initialize() {
val vertexSource = loadShaderSource(R.raw.tiltshift_vertex)
val fragmentSource = loadShaderSource(R.raw.tiltshift_fragment)
val vertexShader = compileShader(GLES20.GL_VERTEX_SHADER, vertexSource)
val fragmentShader = compileShader(GLES20.GL_FRAGMENT_SHADER, fragmentSource)
programId = GLES20.glCreateProgram()
GLES20.glAttachShader(programId, vertexShader)
GLES20.glAttachShader(programId, fragmentShader)
GLES20.glLinkProgram(programId)
// Check for link errors
val linkStatus = IntArray(1)
GLES20.glGetProgramiv(programId, GLES20.GL_LINK_STATUS, linkStatus, 0)
if (linkStatus[0] == 0) {
val error = GLES20.glGetProgramInfoLog(programId)
GLES20.glDeleteProgram(programId)
throw RuntimeException("Shader program link failed: $error")
}
// Get attribute locations
aPositionLocation = GLES20.glGetAttribLocation(programId, "aPosition")
aTexCoordLocation = GLES20.glGetAttribLocation(programId, "aTexCoord")
// Get uniform locations
uTextureLocation = GLES20.glGetUniformLocation(programId, "uTexture")
uModeLocation = GLES20.glGetUniformLocation(programId, "uMode")
uIsFrontCameraLocation = GLES20.glGetUniformLocation(programId, "uIsFrontCamera")
uAngleLocation = GLES20.glGetUniformLocation(programId, "uAngle")
uPositionXLocation = GLES20.glGetUniformLocation(programId, "uPositionX")
uPositionYLocation = GLES20.glGetUniformLocation(programId, "uPositionY")
uSizeLocation = GLES20.glGetUniformLocation(programId, "uSize")
uBlurAmountLocation = GLES20.glGetUniformLocation(programId, "uBlurAmount")
uFalloffLocation = GLES20.glGetUniformLocation(programId, "uFalloff")
uAspectRatioLocation = GLES20.glGetUniformLocation(programId, "uAspectRatio")
uResolutionLocation = GLES20.glGetUniformLocation(programId, "uResolution")
Fix concurrency, lifecycle, performance, and config issues from audit Concurrency & bitmap lifecycle: - Defer bitmap recycling by one cycle so Compose finishes drawing before native memory is freed (preview bitmaps, thumbnails) - Make galleryPreviewSource @Volatile for cross-thread visibility - Join preview job before recycling source bitmap in cancelGalleryPreview() to prevent use-after-free during CPU blur loop - Add @Volatile to TiltShiftRenderer.currentTexCoords (UI/GL thread race) - Fix error dismiss race with cancellable Job tracking Lifecycle & resource management: - Release GL resources via glSurfaceView.queueEvent (must run on GL thread) - Pause GLSurfaceView when entering gallery preview mode - Shut down captureExecutor in CameraManager.release() (thread leak) - Use WeakReference for lifecycleOwnerRef to avoid Activity GC delay - Fix thumbnail bitmap leak on coroutine cancellation (add to finally) - Guarantee imageProxy.close() in finally block Performance: - Compute gradient mask at 1/4 resolution with bilinear upscale (~93% less per-pixel trig work, ~75% less mask memory) - Precompute cos/sin on CPU, pass as uCosAngle/uSinAngle uniforms (eliminates per-fragment transcendental calls in GLSL) - Unroll 9-tap Gaussian blur kernel (avoids integer-branched weight lookup that de-optimizes on mobile GPUs) - Add 80ms debounce to preview recomputation during slider drags Silent failure fixes: - Check bitmap.compress() return value; report error on failure - Log all loadBitmapFromUri null paths (stream, dimensions, decode) - Surface preview computation errors and ActivityNotFoundException to user - Return boolean from writeExifToUri, log at ERROR level - Wrap gallery preview downscale in try-catch (OOM protection) Config: - Add ACCESS_MEDIA_LOCATION permission (GPS EXIF on Android 10+) - Accept coarse-only location grant for geotags - Remove dead adjustResize (no effect with edge-to-edge) - Set windowBackground to black (eliminates white flash on cold start) - Add values-night theme for dark mode - Remove overly broad ProGuard keeps (CameraX/GMS ship consumer rules) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:44:12 +01:00
uCosAngleLocation = GLES20.glGetUniformLocation(programId, "uCosAngle")
uSinAngleLocation = GLES20.glGetUniformLocation(programId, "uSinAngle")
// Clean up shaders (they're linked into program now)
GLES20.glDeleteShader(vertexShader)
GLES20.glDeleteShader(fragmentShader)
}
/**
* Uses the shader program and sets uniforms.
*/
fun use(textureId: Int, params: BlurParameters, width: Int, height: Int, isFrontCamera: Boolean = false) {
GLES20.glUseProgram(programId)
// Bind camera texture
GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId)
GLES20.glUniform1i(uTextureLocation, 0)
// Set effect parameters
GLES20.glUniform1i(uModeLocation, if (params.mode == BlurMode.RADIAL) 1 else 0)
GLES20.glUniform1i(uIsFrontCameraLocation, if (isFrontCamera) 1 else 0)
GLES20.glUniform1f(uAngleLocation, params.angle)
GLES20.glUniform1f(uPositionXLocation, params.positionX)
GLES20.glUniform1f(uPositionYLocation, params.positionY)
GLES20.glUniform1f(uSizeLocation, params.size)
GLES20.glUniform1f(uBlurAmountLocation, params.blurAmount)
GLES20.glUniform1f(uFalloffLocation, params.falloff)
GLES20.glUniform1f(uAspectRatioLocation, params.aspectRatio)
GLES20.glUniform2f(uResolutionLocation, width.toFloat(), height.toFloat())
Fix concurrency, lifecycle, performance, and config issues from audit Concurrency & bitmap lifecycle: - Defer bitmap recycling by one cycle so Compose finishes drawing before native memory is freed (preview bitmaps, thumbnails) - Make galleryPreviewSource @Volatile for cross-thread visibility - Join preview job before recycling source bitmap in cancelGalleryPreview() to prevent use-after-free during CPU blur loop - Add @Volatile to TiltShiftRenderer.currentTexCoords (UI/GL thread race) - Fix error dismiss race with cancellable Job tracking Lifecycle & resource management: - Release GL resources via glSurfaceView.queueEvent (must run on GL thread) - Pause GLSurfaceView when entering gallery preview mode - Shut down captureExecutor in CameraManager.release() (thread leak) - Use WeakReference for lifecycleOwnerRef to avoid Activity GC delay - Fix thumbnail bitmap leak on coroutine cancellation (add to finally) - Guarantee imageProxy.close() in finally block Performance: - Compute gradient mask at 1/4 resolution with bilinear upscale (~93% less per-pixel trig work, ~75% less mask memory) - Precompute cos/sin on CPU, pass as uCosAngle/uSinAngle uniforms (eliminates per-fragment transcendental calls in GLSL) - Unroll 9-tap Gaussian blur kernel (avoids integer-branched weight lookup that de-optimizes on mobile GPUs) - Add 80ms debounce to preview recomputation during slider drags Silent failure fixes: - Check bitmap.compress() return value; report error on failure - Log all loadBitmapFromUri null paths (stream, dimensions, decode) - Surface preview computation errors and ActivityNotFoundException to user - Return boolean from writeExifToUri, log at ERROR level - Wrap gallery preview downscale in try-catch (OOM protection) Config: - Add ACCESS_MEDIA_LOCATION permission (GPS EXIF on Android 10+) - Accept coarse-only location grant for geotags - Remove dead adjustResize (no effect with edge-to-edge) - Set windowBackground to black (eliminates white flash on cold start) - Add values-night theme for dark mode - Remove overly broad ProGuard keeps (CameraX/GMS ship consumer rules) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:44:12 +01:00
// Precompute angle trig on CPU to avoid per-fragment transcendental calls.
// The adjusted angle accounts for the 90deg coordinate transform.
val adjustedAngle = if (isFrontCamera) {
-params.angle - (Math.PI / 2).toFloat()
} else {
params.angle + (Math.PI / 2).toFloat()
}
GLES20.glUniform1f(uCosAngleLocation, cos(adjustedAngle))
GLES20.glUniform1f(uSinAngleLocation, sin(adjustedAngle))
}
/**
* Releases shader resources.
*/
fun release() {
if (programId != 0) {
GLES20.glDeleteProgram(programId)
programId = 0
}
}
private fun loadShaderSource(resourceId: Int): String {
val inputStream = context.resources.openRawResource(resourceId)
val reader = BufferedReader(InputStreamReader(inputStream))
return reader.use { it.readText() }
}
private fun compileShader(type: Int, source: String): Int {
val shader = GLES20.glCreateShader(type)
GLES20.glShaderSource(shader, source)
GLES20.glCompileShader(shader)
// Check for compile errors
val compileStatus = IntArray(1)
GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compileStatus, 0)
if (compileStatus[0] == 0) {
val error = GLES20.glGetShaderInfoLog(shader)
GLES20.glDeleteShader(shader)
val shaderType = if (type == GLES20.GL_VERTEX_SHADER) "vertex" else "fragment"
throw RuntimeException("$shaderType shader compilation failed: $error")
}
return shader
}
}