如何跟踪头部位置

本文关键字:头部 位置 跟踪 何跟踪 | 更新日期: 2023-09-27 18:02:33

我想做一些类似于johnny Lee在他的Wii头部跟踪中所做的事情http://www.youtube.com/watch?v=Jd3-eiid-Uw&特性= player_embedded

但是我想用Kinect。由于微软的sdk暴露了骨骼关节,我希望我可以使用它来获得头部位置。问题是,我想在我的台式电脑和显示器上做到这一点。如果我把Kinect传感器放在显示器旁边,然后坐在桌子旁。几乎只有我的头和脖子可以被传感器看到,所以骨骼跟踪不会捕捉到我的头部位置。

有人熟悉使用Kinect的头部跟踪项目吗?最好使用c#

如何跟踪头部位置

我认为对于这个应用程序,你不能使用任何框架提供的骨架跟踪,如微软的SDK或OpenNI。

我建议通过对原始深度数据应用深度阈值来分割用户头部。这将导致背景缩小。我认为已经有现成的方法可以做到这一点。

作为第二步,你想在分割的用户内部有一个类似轴的东西。最简单的方法是使用opencv fitEllipse。返回的椭圆的长轴结合深度信息给出了这个轴。

这种方法只适用于大多数分割点属于用户头部的情况。如果你在更远的地方,你必须考虑一种只分割头部的方法。

Windows SDK的官方Kinect有限制,它符合为XBox和XDK提供的指导,因为您需要距离传感器1.2m到3.5m才能使用Kinect传感器。这种限制实际上在OpenNI/NITE库等替代sdk中有所减少,这些sdk允许您检测离传感器更近的骨架/对象。

骷髅输入的问题是它只会检测头部与骨骼的比例,但如果你左右旋转头部则不会。为了实现这一点,你不会使用原始深度流和一些围绕物体识别的智能,这有点复杂。

在过去,我使用这个商业。net API,它使用一个网络摄像头来跟踪头部运动,并实现你想要的:http://luxand.com/facesdk/index2.php

你不需要kinect来追踪头部位置。你可以用普通相机和openCV做同样的事情,利用面部追踪。

下面是一个简单的例子:http://vimeo.com/19464641

在视频中,我使用openCV来跟踪我的脸(你几乎看不到角落,但红点表示我的脸位置)。

查看第九频道关于这类主题的教程。你会去看骨骼基础视频。但是如果你想节省时间,这里有一些代码。
XAML

<Window x:Class="SkeletalTracking.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="MainWindow" Height="600" Width="800" Loaded="Window_Loaded" 
    xmlns:my="clr-namespace:Microsoft.Samples.Kinect.WpfViewers;assembly=Microsoft.Samples.Kinect.WpfViewers" 
    Closing="Window_Closing" WindowState="Maximized">       
<Canvas Name="MainCanvas">
    <my:KinectColorViewer Canvas.Left="0" Canvas.Top="0" Width="640" Height="480" Name="kinectColorViewer1" 
                          Kinect="{Binding ElementName=kinectSensorChooser1, Path=Kinect}" />
    <my:KinectSensorChooser Canvas.Left="250" Canvas.Top="380" Name="kinectSensorChooser1" Width="328" />
    <Image Canvas.Left="66" Canvas.Top="90" Height="87" Name="headImage" Stretch="Fill" Width="84" Source="/SkeletalTracking;component/c4f-color.png" />
</Canvas>

内部代码

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using Microsoft.Kinect;
using Coding4Fun.Kinect.Wpf; 
namespace SkeletalTracking
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
    }
    bool closing = false;
    const int skeletonCount = 6; 
    Skeleton[] allSkeletons = new Skeleton[skeletonCount];
    private void Window_Loaded(object sender, RoutedEventArgs e)
    {
        kinectSensorChooser1.KinectSensorChanged += new DependencyPropertyChangedEventHandler(kinectSensorChooser1_KinectSensorChanged);
    }
    void kinectSensorChooser1_KinectSensorChanged(object sender, DependencyPropertyChangedEventArgs e)
    {
        KinectSensor old = (KinectSensor)e.OldValue;
        StopKinect(old);
        KinectSensor sensor = (KinectSensor)e.NewValue;
        if (sensor == null)
        {
            return;
        }


        var parameters = new TransformSmoothParameters
        {
            Smoothing = 0.3f,
            Correction = 0.0f,
            Prediction = 0.0f,
            JitterRadius = 1.0f,
            MaxDeviationRadius = 0.5f
        };
        //sensor.SkeletonStream.Enable(parameters);
        sensor.SkeletonStream.Enable();
        sensor.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(sensor_AllFramesReady);
        sensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30); 
        sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
        try
        {
            sensor.Start();
        }
        catch (System.IO.IOException)
        {
            kinectSensorChooser1.AppConflictOccurred();
        }
    }
    void sensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
    {
        if (closing)
        {
            return;
        }
        //Get a skeleton
        Skeleton first =  GetFirstSkeleton(e);
        if (first == null)
        {
            return; 
        }

        //set scaled position
        ScalePosition(headImage, first.Joints[JointType.Head]);
        //ScalePosition(leftEllipse, first.Joints[JointType.HandLeft]);
        //ScalePosition(rightEllipse, first.Joints[JointType.HandRight]);
        GetCameraPoint(first, e); 
    }
    void GetCameraPoint(Skeleton first, AllFramesReadyEventArgs e)
    {
        using (DepthImageFrame depth = e.OpenDepthImageFrame())
        {
            if (depth == null ||
                kinectSensorChooser1.Kinect == null)
            {
                return;
            }

            //Map a joint location to a point on the depth map
            //head
            DepthImagePoint headDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.Head].Position);
            //left hand
            DepthImagePoint leftDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandLeft].Position);
            //right hand
            DepthImagePoint rightDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandRight].Position);

            //Map a depth point to a point on the color image
            //head
            ColorImagePoint headColorPoint =
                depth.MapToColorImagePoint(headDepthPoint.X, headDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //left hand
            ColorImagePoint leftColorPoint =
                depth.MapToColorImagePoint(leftDepthPoint.X, leftDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //right hand
            ColorImagePoint rightColorPoint =
                depth.MapToColorImagePoint(rightDepthPoint.X, rightDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);

            //Set location
            CameraPosition(headImage, headColorPoint);
            //CameraPosition(leftEllipse, leftColorPoint);
            //CameraPosition(rightEllipse, rightColorPoint);
        }        
    }

    Skeleton GetFirstSkeleton(AllFramesReadyEventArgs e)
    {
        using (SkeletonFrame skeletonFrameData = e.OpenSkeletonFrame())
        {
            if (skeletonFrameData == null)
            {
                return null; 
            }

            skeletonFrameData.CopySkeletonDataTo(allSkeletons);
            //get the first tracked skeleton
            Skeleton first = (from s in allSkeletons
                                     where s.TrackingState == SkeletonTrackingState.Tracked
                                     select s).FirstOrDefault();
            return first;
        }
    }
    private void StopKinect(KinectSensor sensor)
    {
        if (sensor != null)
        {
            if (sensor.IsRunning)
            {
                //stop sensor 
                sensor.Stop();
                //stop audio if not null
                if (sensor.AudioSource != null)
                {
                    sensor.AudioSource.Stop();
                }

            }
        }
    }
    private void CameraPosition(FrameworkElement element, ColorImagePoint point)
    {
        //Divide by 2 for width and height so point is right in the middle 
        // instead of in top/left corner
        Canvas.SetLeft(element, point.X - element.Width / 2);
        Canvas.SetTop(element, point.Y - element.Height / 2);
    }
    private void ScalePosition(FrameworkElement element, Joint joint)
    {
        //convert the value to X/Y
        //Joint scaledJoint = joint.ScaleTo(1280, 720); 
        //convert & scale (.3 = means 1/3 of joint distance)
        //Joint scaledJoint = joint.ScaleTo(1280, 720, .3f, .3f);
        Canvas.SetLeft(element, scaledJoint.Position.X);
        Canvas.SetTop(element, scaledJoint.Position.Y); 
    }

    private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
    {
        closing = true; 
        StopKinect(kinectSensorChooser1.Kinect); 
    }

   }
}

我个人建议看视频,因为它们解释了一切。祝你的项目好运!

我建议使用:Aforge.net与微软XNA框架或单独使用Aforge.net。不过你需要自己做一些开发工作。我也在用c#做类似的事情。我想你不可能找到一个完整的现成的例子。还没有人这么做。(如果我说错了请指正)。