Parametrized Proxy Model:

An articulated, parametric human model is used as the proxy model. The joint angles and bone lengths are the free variables of the model and serve as the parameters using which any position of the actor can be represented. The parameters change over time, but are same for all views at any time instant. We use an articulated model with 18 joints. Each joint has 3 angles for heading, pitch, and roll. We currently use a single size parameter to scale the base model uniformly. Thus, the model is defined completely using only 55 parameters in each frame. The basic model is skeletal. A triangulated skin-model is attached to it and provides a realistic human model. The basic skeletal and skinned models are available to both the encoder and the decoder. Thus, the 55 parameters provide a decent prediction for the common geometric structure of the actor for each frame.

We compute the parameters of the proxy by first unprojecting each depth map from each view to get a point cloud in the world. The problem of fitting an articulated model from images and from depths has been studied in the past [13,3,1]. The fitting can be performed by optimizing the error between the skinned model and the point cloud. Fitting subsequent frames is easier as the parameters change slowly. Since the model fitting is not our main focus, we fit the model interactively using a semi-automatic tool built for the purpose to get a basic proxy with minimal efforts. In the end, the parameters of the articulated model for each frame represent the scene parametrically as a proxy.

Figure 4: Decoding of residues at the Client