How We Accelerated Feature Hashing for Ad Ranking on GPUs

Background

In ad‑ranking (coarse‑ranking) the model is shallow but consumes a huge number of features, so feature parsing and embedding dominate the latency. The baseline implementation assembled data on the CPU, then copied it to GPU for hash, with the CPU step taking over 90% of the hash time.

Integer Feature Hash Optimization

Because integer features are first converted to strings (AsString) before hashing, the string Tensor created on the CPU is highly fragmented, causing costly host‑to‑device copies. To avoid this, we moved the entire int‑to‑string and hash pipeline to the GPU.

Scheme 1 – Parallel CPU Compute & Host‑to‑Device Copy

Copy the int64 tensor to GPU memory while the CPU simultaneously computes each element’s size and offset. The GPU kernel then converts integers to strings in a contiguous buffer and performs hashing using the pre‑computed offsets.

Scheme 2 – Full GPU Computation

All size calculations and integer‑to‑string (itoa) conversions are performed on the GPU, eliminating CPU bottlenecks. Two methods were explored:

Remainder‑based conversion that writes characters into a thread‑local buffer.

Pre‑computing decimal length using logarithmic bounds to avoid per‑character loops.

After benchmarking, the remainder‑based method (Scheme 2.1) was chosen for production.

Custom String Tensor

We introduced a custom

ConsecString

type stored in a

DynamicBuffer

and wrapped it in TensorFlow’s

Variant

. This yields a contiguous string tensor that can be copied to GPU memory in one shot, removing the fragmented copies required by the original string Tensor.

<code>struct ConsecString {
  DynamicBuffer* buf;
  explicit ConsecString();
  explicit ConsecString(DynamicBuffer* buf);
  void Encode(VariantTensorData* data) const;
  bool Decode(const VariantTensorData&amp; data);
  string TypeName() const {return "ConsecString";}
  string DebugString() const {return "DebugString: ConsecString";}
};

struct DynamicBuffer {
  std::vector&lt;string&gt; buf_list;
  std::vector&lt;std::vector&lt;int32&gt;&gt; sizes_list;
  void merge();
  void reset();
};
</code>

Performance Results

Running the optimized pipeline on a V100 GPU showed dramatic reductions in end‑to‑end latency. For example, processing 1 000 000 features dropped from 189 s (baseline) to 15.14 s (optimized), a 1248× speed‑up. Similar gains were observed across various batch sizes and feature counts.

Additional micro‑benchmarks on a single feature’s hash operation demonstrated speed‑ups ranging from 103% to over 1200% depending on the batch size.

Key Takeaways

Moving integer‑to‑string conversion and hash computation to the GPU, combined with a custom contiguous string tensor, eliminates fragmented memory copies and fully utilizes GPU parallelism, yielding order‑of‑magnitude performance improvements for large‑scale ad‑ranking workloads.