An Adaptive Superpixel Based Hand Gesture Tracking and Recognition System

Algorithm 2

Adaptive superpixel based hand gesture tracking.

For each frame 𝑡=𝑀+1 to the end

Normal hand tracking

Input: frame 𝐼_𝑡,𝑋_𝑡−1

(1)  SLIC get 𝑃 superpixels 𝑆_𝑝 on surrounding of 𝑋_𝑡−1.

(2)  For each superpixel 𝑟, Compute theYCbCr histgram 𝑓^𝑟_𝑡, and confidence map 𝐶^𝑠_𝑟 using (6).

(3)  Sample 𝑁 candidates 𝑋^𝑛_𝑡^𝑁_𝑛=1 around 𝑋_𝑡−1 with 𝐶^𝑠_𝑟, discard unproper samples using (7) and (8).

(4)  Calculate the motion parameter 𝑝(𝑋^𝑛_𝑡∣𝑋_𝑡−1) for each 𝑋^𝑛_𝑡 using (9).

(5)  Calculate the likelihood 𝐶^𝑛_𝑡 for each 𝑋^𝑛_𝑡 using (10).

(6)  Get the best match of hand 𝑋_𝑡 with MAP estimate on 𝑝(𝑋^𝑛_𝑡∣𝑋_𝑡−1) and 𝐶^𝑛_𝑡 using (11).

Output: hand detection 𝑋_𝑡 in frame t.

Failure recovery and updating

Input: current hand detection 𝑋_𝑡

(1)  Check the occurance of occlusion with threshold. Calculate 𝑀×𝑁 similarity matrix using (4). Detect the hand location 𝑋_𝑡 to recover the occlusion.

(2)  Check the occurance of background confusion using (12) and re-track 𝐾 frames to recover 𝑋_𝑘.

(3)  In case of background confusion, sample all detections of re-tracking (𝑈=1) and discard all previous samples.

(4)  In case of occlusion or no failure, use one sample for every 𝑈 frames to replace a previous sample using (13).

(5)  Replace the appearance model every 𝑊 frames by re-train on new samples.

Output: recovered hand detection 𝑋_𝑡 if normal hand tracking fails, and new hand appearance model 𝑀_𝑎.