PowerImage: Integrating FFI and External Texture for Advanced Flutter Image Handling

Background : The previous Xianyu image library performed well at scale but faced issues such as cache inefficiency when mixing native images, inability to display images on simulators, and lack of custom image channels.

To address these, the team introduced PowerImage , a new solution that combines external texture with FFI to provide native‑like capabilities.

Core Capabilities :

Support loading ui.Image directly.

Enable image pre‑caching similar to precacheImage .

Unified texture cache with native image cache to avoid memory duplication.

Simulator support for Flutter versions ≤ 1.23.0‑18.1.pre.

Custom image‑type channels for business‑specific needs.

Comprehensive exception capture.

Animated‑image support (in progress).

Flutter Native Image Flow : The native Image widget obtains an ImageStream from an ImageProvider , uses frameBuilder , loadingBuilder , and finally builds a RawImage which renders via RenderImage . The key data resides in ImageInfo.ui.Image .

New Architecture :

FFI path: native side extracts raw pixel data (e.g., iOS code below) and passes address, length, width, height, rowBytes, and pixel format to Dart.

Dart side reconstructs a ui.Image using decodeImageFromPixels and releases native memory after use.

_rowBytes = CGImageGetBytesPerRow(cgImage); CGDataProviderRef dataProvider = CGImageGetDataProvider(cgImage); CFDataRef rawDataRef = CGDataProviderCopyData(dataProvider); _handle = (long)CFDataGetBytePtr(rawDataRef); NSData *data = CFBridgingRelease(rawDataRef); self.data = data; _length = data.length;

Dart implementation:

@override FutureOr createImageInfo(Map map) { Completer completer = Completer (); int handle = map['handle']; int length = map['length']; int width = map['width']; int height = map['height']; int rowBytes = map['rowBytes']; ui.PixelFormat pixelFormat = ui.PixelFormat.values[map['flutterPixelFormat'] ?? 0]; Pointer pointer = Pointer .fromAddress(handle); Uint8List pixels = pointer.asTypedList(length); ui.decodeImageFromPixels(pixels, width, height, pixelFormat, (ui.Image image) { ImageInfo imageInfo = ImageInfo(image: image); completer.complete(imageInfo); PowerImageLoader.instance.releaseImageRequest(options); }, rowBytes: rowBytes); return completer.future; }

Two optimization directions were explored:

Let Flutter decode the image to avoid an extra memory copy.

Collaborate with the Flutter team to reduce internal copies.

Texture Approach : Uses a custom TextureImage class that implements ui.Image solely for cache size calculation.

import 'dart:typed_data'; import 'dart:ui' as ui show Image; import 'dart:ui'; class TextureImage implements ui.Image { int _width; int _height; int textureId; TextureImage(this.textureId, int width, int height) : _width = width, _height = height; @override void dispose() {} @override int get height => _height; @override Future toByteData({ImageByteFormat format = ImageByteFormat.rawRgba}) { throw UnimplementedError(); } @override int get width => _width; }

ImageCache extension for Flutter < 2.2.0> adds removal callbacks:

typedef void HasRemovedCallback(dynamic key, dynamic value); class RemoveAwareMap implements Map { HasRemovedCallback hasRemovedCallback; // ... } final RemoveAwareMap _pendingImages = RemoveAwareMap (); void hasImageRemovedCallback(dynamic key, dynamic value) { if (key is ImageProviderExt) { waitingToBeCheckedKeys.add(key); } if (isScheduledImageStatusCheck) return; isScheduledImageStatusCheck = true; scheduleMicrotask(() { waitingToBeCheckedKeys.forEach((key) { if (!_pendingImages.containsKey(key) && !_cache.containsKey(key) && !_liveImages.containsKey(key)) { if (key is ImageProviderExt) { key.dispose(); } } }); waitingToBeCheckedKeys.clear(); isScheduledImageStatusCheck = false; }); }

Performance Comparison (iPhone 11 Pro, 300 network images):

Texture memory peak ~395 MB, smoother.

FFI memory peak ~480 MB, noticeable spikes due to copy.

UI‑thread timing: Texture fastest, PowerImage slightly better than IFImage. Raster‑thread: PowerImage outperforms IFImage; native approach wins because it resizes images.

Conclusions :

Prefer Texture for everyday scenarios.

Use FFI when: Running on Flutter ≤ 1.23.0‑18.1.pre on simulators. Need direct ui.Image data. Flutter‑side decoding is acceptable (e.g., custom decoding libraries).

Future Work : Animated‑image support will replace OneFrameImageStreamCompleter with MultiFrameImageStreamCompleter . The team plans to open‑source PowerImage on XianyuTech by the end of the year, providing design docs, integration guides, performance reports, and unit tests.