AI and Front‑End Integration: Insights from the 58 Group Technical Salon

On May 27, 2019, the 58 Group Technical Salon (Session 12) was held at the Beijing headquarters, featuring a front‑end and AI theme. Xiaomi deep‑learning group algorithm expert Guo Chengkai and 58 front‑end senior architect Sun Dong shared their experiences on applying deep learning to graphic animation and implementing AI on the front‑end.

1. Xiaomi Deep Learning in Graphic Animation and Front‑End Landing

Guo Chengkai introduced Xiaomi AI applications such as facial transfer, motion transfer, portrait stitching, parameter face swapping, style transfer, doodle creation, generative editing, and intelligent editing, which are already deployed in Xiaomi camera, scene camera, and voice assistant products.

Model Training Elements

Key factors include high‑quality, diverse datasets, appropriate loss functions, convolutional networks, feature compression, key‑point detection, and differentiable generative networks.

Optimization Strategies

Hardware acceleration using Qualcomm Snapdragon 845 (Adreno 630) improves graphics throughput 2.5× while reducing power consumption 30%. Model acceleration is achieved by factorizing standard convolutions into depthwise and 1×1 convolutions, reducing parameters and FLOPs, enabling faster inference and easier parallelism. Grouped convolutions similar to ResNeXt further cut computation and parameters with minimal accuracy loss.

2. 58 AI and Front‑End Practice

58’s AI applications include intelligent客服, writing assistants, Q&A bots, voice bots, and more. To bring AI to the browser, three workflows are considered:

Import a pre‑trained model for inference after converting it to a web‑compatible format.

Fine‑tune the imported model using transfer learning with a small amount of browser‑collected data.

Define, train, and run a model entirely within the browser.

The team focuses on the first workflow, leveraging TensorFlow and TensorFlow.js. After converting TensorFlow/Keras models to web‑model files, they can be loaded in the browser for prediction without any additional libraries or drivers.

Scenario Landing

AI brings unique advantages to front‑end scenarios such as highly interactive interfaces, image beautification and recognition, video face detection, and design tools like pix2code or sketch2react. Specific use cases explored include AI photo enhancement, image classification, AR video capture, and Word2Vec semantic analysis.

Using TensorFlow.js and the converter, models are transformed into web‑compatible formats, enabling low‑latency, offline image processing directly in the browser.

3. Detail Optimizations

Challenges include limited support in tensorflowjs‑converter (e.g., missing encode/decode ops) and increased model size after conversion. Lazy‑loading and pre‑loading strategies are employed to mitigate the impact of numerous small weight files on initial load time.

Summary

The salon highlighted the importance of model size reduction and quality for successful front‑end AI deployment. Guo Chengkai’s insights on training tricks, model compression, and optimization for mobile devices provide valuable guidance for future front‑end AI projects.

Outlook

While full model training in the browser remains limited by computational resources, ongoing improvements in JavaScript engines and AI frameworks (e.g., TensorFlow.js with WebGL support) suggest that simpler algorithms will soon run entirely on the client, reducing server load and enabling new interactive machine‑learning experiences.