An Adaptive Superpixel Based Hand Gesture Tracking and Recognition System

Algorithm 1

Initial hand detection and model construction.

Initial hand detection

Input: M frames 𝐼_𝑖∈ℝ^𝐻×𝑊,𝑖∈[1,𝑀]

(1)  Segment 𝐼₁ into 𝑃 superpixels 𝑆_𝑝, (𝑝=1, …, 𝑃) with SLIC.

(2)  For each frame 𝐼=𝐼₂ to 𝐼_𝑀, Compute 𝐷_𝑝 for each 𝑆_𝑝 using (1).

(3)  Detect 𝑚 superpixels 𝑃_𝑚 (𝑚<𝑃) with motions using (2), merge neighbored superpixels to get 𝑅 regions.

(4)  Do CTM segmentation on surrounding of 𝑅 regions in 𝐼₁, get object regions 𝑂₁, …,𝑂_𝐾 with area 𝐴₁, …, 𝐴_𝐾.

(5)  Find the hand region from 𝑅 and 𝑂_𝑘 regions using (3).

Output: 𝑋₁ (center of hand region and its scale in the first frame).

Hand appearance model construction

Input: 𝑀 frames 𝐼_𝑖∈ℝ^𝐻×𝑊,𝑖∈[1,𝑀] and 𝑋₁

(1)  For each frame 𝐼_𝑡, 𝑡=2, …, 𝑀, detect the hand 𝑋_𝑡 from 𝑁 candidates around 𝑋_𝑡−1 using (4).

(2)  For each frame 𝐼_𝑡, 𝑡=1, …, 𝑀, Extract 𝑓^𝑟_𝑡^𝑃_𝑟=1 as the histogram in 𝑌𝐶𝑏𝐶𝑟 of 𝑃 superpixels from SLIC

   segmentation on surronding of 𝑋_𝑡.

(3)  Apply mean shift clustering on feature set 𝐹=𝑓^𝑟_𝑡∣𝑡=1,…,𝑚;𝑟=1,…,𝑃_𝑡 to get 𝑓_𝑐(𝑖),𝑟_𝑐(𝑖) and {𝑓^𝑟_𝑡∣𝑓^𝑟_𝑡∈𝑖}.

   Calclute 𝐶^𝑐_𝑖 using (5).

Output: Hand appearance model 𝑀_𝑎={𝐶^𝑐_𝑖,𝑓_𝑐(𝑖),𝑟_𝑐(𝑖),{𝑓^𝑟_𝑡∣𝑓^𝑟_𝑡∈𝑖}}.