首页 > 代码库 > 微软Hololens学院教程-Hologram 230-空间映射(Spatial mapping )
微软Hololens学院教程-Hologram 230-空间映射(Spatial mapping )
空间映射地图是将真实环境的环境信息扫描到设备中,使得全息对象可以识别真实场景环境,从而达到可以将虚拟对象与真实世界相结合的效果。这节教程主要学习内容如下:
- 使用Hololens扫描空间环境并将空间数据导入到开发计算机中。
- 学习利用shader给空间网格赋予材质以便其更容易被发现。
- 使用网格处理方法将网格变成简单的平面。
- 对全息对象可以放置的位置进行放置提醒,使得用户更容易的放置。
- 开发遮挡效果,即当全息对象被真实场景中的物体遮挡时,你仍然可以看见它,只不过它是线框模式的。
项目文件:
Download the files required by the project.
Unity 设置
- 打开 Unity.
- 选择新建 New创建一个新的项目.
- 将项目命名为 Planetarium.
- 保存地址到你下载的项目文件夹下 HolographicAcademy-Holograms-230-SpatialMapping .
- 选择 3D .
- 点击创建项目 Create Project.
- 当Unity打开后,打开 Edit > Project Settings > Player.
- 在Inspector 面板下,点击选择 Windows Store 图标.
- 展开 Other Settings 设置.
- 在Rendering 部分, 勾选 Use 16-bit Depth Buffers 选项.
- 在Rendering部分, 勾选Virtual Reality Supported 选项.
- 确保Windows Holographic 出现在 Virtual Reality SDKs 列表中. 如果没有,选择 + 按钮再选上 Windows Holographic.
- 展开 Publishing Settings .
- 再 Capabilities 部分,勾选以下设置:
- InternetClientServer
- PrivateNetworkClientServer
- Microphone
- SpatialPerception
- 到 Edit > Project Settings > Quality
- 在 Inspector 面板下, 点击 Windows Store 图标最下方, ‘Default‘ 的第二个下拉黑色箭头,将默认设置改为 Fastest.
- 到project面板下, Assets > Import Package > Custom Package.
- 定位到你下载的项目文件...\HolographicAcademy-Holograms-230-SpatialMapping\Starting 文件夹.
- 点击 Planetarium.unitypackage.
- 点击Open.
- 一个Import Unity Package窗口会出现,点击Import 按钮.
- 等待Unity导入所有的项目资源.
- 在Hierarchy 面板中,删除 Main Camera.
- 在 Project 面板下, HoloToolkit\Utilities\Prefabs 文件夹,找到 Main Camera 对象.
- 拖拽 Main Camera 预设到 Hierarchy 面板中.
- 在 Hierarchy 面板中,删除 Directional Light .
- 在 Project 面板, Holograms 文件夹下,定位到 Cursor 对象.
- 拖拽 Cursor 预设到 Hierarchy 面板中.
- 选中 Hierarchy 面板下的 Cursor 对象.
- 在Inspector 面板下,点击 Layer 下拉按钮选择 Edit Layers....
- 将 User Layer 31 命名为 "SpatialMapping" 并添加此标签.
- 将当前场景保存: File > Save Scene As...
- 点击 New Folder 创建一个新的文件夹为 Scenes.
- 命名新场景为 "Planetarium" 然后将它保存在 Scenes 文件夹下.
章节1 扫描
步骤:
- 在 Project 面板下打开HoloToolkit\SpatialMapping\Prefabs 文件夹, 找到 SpatialMapping 预设.
- 拖拽 SpatialMapping 预设到 Hierarchy 面板中.
发布测试 (part 1)
- 在 Unity中选择 File > Build Settings.
- 点击 Add Open Scenes 将当前场景 Planetarium 添加到列表中.
- 选择Platform 列表下的 Windows Store 然后点击下方的 Switch Platform.
- 设置 SDK 为 Universal 10 ,设置 UWP Build Type 为 D3D.
- 勾选 Unity C# Projects.
- 点击 Build.
- 创建一个新文件 New Folder 命名 "App".
- 单机App 文件夹.
- 点击 Select Folder 按钮.
- 当Unity发布完成,一个文件资源管理器窗口会弹出.
- 双击 App 文件夹并打开它.
- 双击 Planetarium.sln 用VS打开.
- 在VS中,将顶部工具栏内设置改为 Release,x86.
- 选择远程设备Remote Machine.
- 输入你设备的IP地址 your device‘s IP address ,选择通用未加密模式Universal (Unencrypted Protocol).记得设备要连网。
- 点击Debug -> Start Without debugging 或者 Ctrl + F5.
- 当你的APP成功发布到你的Hololens上时. 戴上它在你的房间行走,你会看到你房间的物体表面会被黑色和白色线框网格覆盖。
- 扫描你的房间. 确保你看到了墙面,地面,以及天花板。
发布测试 (part 2)
- 在unity 中,选择分析器Window > Profiler.
- 点击添加GPU分析器 Add Profiler > GPU.
- 点击激活分析器 Active Profiler > <Enter IP>.
- 输入你设备的IP地址 IP address .
- 点击连接 Connect.
- 观察GPU渲染帧所需的毫秒数.
- 在设备上关闭应用.
- 返回到VS中,打开 SpatialMappingObserver.cs 脚本. 你会在Assembly-CSharp (Universal Windows) 项目的HoloToolkit\SpatialMapping 文件夹里找到它 .
- 定位到 Awake() 函数,添加如下代码: TrianglesPerCubicMeter = 1200;
- 重新部署项目到你的设备上,然后重新连接分析器 profiler. 观察GPU渲染帧所需的毫秒数的变化.
- 停止运行应用.
将扫描数据保存在电脑中
接下来需要将扫描的空间数据下载下来并导入Unity中以供编辑。
- 返回到VS中,将上一步在Awake() 函数中添加的 TrianglesPerCubicMeter 代码删除.
- 从新将项目发布到你的设备上. 我们现在应该运行渲染的是每立方米500个三角形。
- 在电脑上打开你的浏览器,输入https://<你的IP地址> ,可以打开设备控制台,Windows Device Portal.
- 选择 3D View .带上你的设备扫描室内一周
- 选择Update 按钮.
- 你会看到你扫描的空间场景会显示在窗口中.
- 点击 Save 按钮.
- 打开你的下载文件夹查看刚下载的空间文件 SRMesh.obj(或者你命名的其他文件).
- 复制SRMesh.obj 到你Unity项目的 Assets 文件夹下.
- 回到Unity, 选择Hierarchy 面板下的 SpatialMapping 对象.
- 在Inspector面板中,找到 Object Surface Observer (Script) 组件.
- 点击 Room Model 属性右边的小圆圈.
- 在弹出窗口中搜索SRMesh 对象并选中,然后关闭窗口.
- 现在 Room Model 属性值为 SRMesh.
- 点击 Play 按钮进入Unity预览模式.
- SpatialMapping 组件会默认加载保存的房间模型的网格数据。
- 选择 Scene 界面来查看你扫描的房间模型.
- 再一次点击 Play 按钮可以退出预览.
NOTE: 下次您在Unity中进入预览模式时,它将默认加载保存的房间网格。
章节2 可视化
给房间网格模型添加材质使其可见。
步骤:
- 在 Unity的 Hierarchy 面板中,选择 SpatialMapping 对象.
- 在 Inspector 面板中找到 Spatial Mapping Manager (Script) 组件.
- 点击Surface Material 属性右边的圆圈.
- 在新弹出的窗口中搜索 BlueLinesOnWalls 材质并选中然后关闭窗口.
- 在 Project 面板下的 Shaders 文件夹,双击 BlueLinesOnWalls用VS打开.
- 这是一个简单的顶点片段着色器程序, 实现了以下几点:
- 将顶点位置转换为世界空间。
- 检查顶点法线以确定像素是否是垂直的.
- 设置要渲染的像素的颜色.
部署发布:
- 返回到Unity,进入预览模式。
- 蓝线将在房间网格的所有垂直表面上呈现。
- 切换到“Scene”选项卡以调整你观看房间的视角,并查看整个房间网格在Unity中的显示方式。
- 在“Project”面板中,找到“Materials”文件夹,然后选择BlueLinesOnWalls材质球。
- 修改材质球的一些属性,并查看更改在Unity编辑器中显示的效果。
-在“Inspector”面板中,调整LineScale值,使线条看起来更厚或更薄。
-在“Inspector”面板中,调整LinesPerMeter值以更改每个墙上显示的行数。 - 退出预览模式。
- 重新部署到HoloLens,并观察着色渲染如何在真实环境表面上显示。
- Unity在渲染材质方面做得很好,但是在设备中不使用材质渲染总是一个好主意。
章节3 处理
这章主要学习处理空间映射数据的技术以在应用程序中使用。分析空间映射数据以查找平面和删除三角形。使全息图正确放置在真实环境中的平面上。
步骤:
- 在 Unity的 Project 面板中, Holograms 文件夹下找到 SpatialProcessing 对象.
- 拖拽 SpatialProcessing 对象到 Hierarchy 面板中.
SpatialProcessing 预制包括了用于处理空间映射数据的组件。 SurfaceMeshesToPlanes.cs脚本将基于空间映射数据查找和生成平面。 在我们的应用程序中我们将使用平面来代表墙壁,地板和天花板。 此预制还包括RemoveSurfaceVertices.cs脚本,它可以从空间映射网格中删除顶点。 这可以用于在网格中创建孔,或者删除不再需要的多余三角形(因为可以使用平面来代替)。
- 在Unity的Project 面板, Holograms 文件夹下找到SpaceCollection 对象.
- 拖拽 SpaceCollection 对象到 Hierarchy 面板中.
- 在 Hierarchy 面板中选择 SpatialProcessing 对象.
- 在 Inspector 面板找到 Play Space Manager (Script) 组件.
- 双击 PlaySpaceManager.cs 脚本用VS打开.
PlaySpaceManager.cs脚本包含特定于应用程序的代码。 我们将向此脚本添加功能以实现以下行为:
1 在超过扫描时间限制(10秒)后停止收集空间映射数据。
2 处理空间映射数据:
- 使用SurfaceMeshesToPlanes创建一个更简单的平面世界(墙壁,地板,天花板等)。
- 使用RemoveSurfaceVertices删除落在平面边界内的表面三角形。
3 生成一个全息图集合,并将它们放置在用户附近的墙壁和地板上。
完成代码:
using System.Collections.Generic; using UnityEngine; using UnityEngine.Windows.Speech; using HoloToolkit.Unity; /// <summary> /// The SurfaceManager class allows applications to scan the environment for a specified amount of time /// and then process the Spatial Mapping Mesh (find planes, remove vertices) after that time has expired. /// </summary> public class PlaySpaceManager : Singleton<PlaySpaceManager> { [Tooltip("When checked, the SurfaceObserver will stop running after a specified amount of time.")] public bool limitScanningByTime = true; [Tooltip("How much time (in seconds) that the SurfaceObserver will run after being started; used when ‘Limit Scanning By Time‘ is checked.")] public float scanTime = 30.0f; [Tooltip("Material to use when rendering Spatial Mapping meshes while the observer is running.")] public Material defaultMaterial; [Tooltip("Optional Material to use when rendering Spatial Mapping meshes after the observer has been stopped.")] public Material secondaryMaterial; [Tooltip("Minimum number of floor planes required in order to exit scanning/processing mode.")] public uint minimumFloors = 1; [Tooltip("Minimum number of wall planes required in order to exit scanning/processing mode.")] public uint minimumWalls = 1; /// <summary> /// Indicates if processing of the surface meshes is complete. /// </summary> private bool meshesProcessed = false; /// <summary> /// GameObject initialization. /// </summary> private void Start() { // Update surfaceObserver and storedMeshes to use the same material during scanning. SpatialMappingManager.Instance.SetSurfaceMaterial(defaultMaterial); // Register for the MakePlanesComplete event. SurfaceMeshesToPlanes.Instance.MakePlanesComplete += SurfaceMeshesToPlanes_MakePlanesComplete; } /// <summary> /// Called once per frame. /// </summary> private void Update() { // Check to see if the spatial mapping data has been processed // and if we are limiting how much time the user can spend scanning. if (!meshesProcessed && limitScanningByTime) { // If we have not processed the spatial mapping data // and scanning time is limited... // Check to see if enough scanning time has passed // since starting the observer. if (limitScanningByTime && ((Time.time - SpatialMappingManager.Instance.StartTime) < scanTime)) { // If we have a limited scanning time, then we should wait until // enough time has passed before processing the mesh. } else { // The user should be done scanning their environment, // so start processing the spatial mapping data... /* TODO: 3.a DEVELOPER CODING EXERCISE 3.a */ // 3.a: Check if IsObserverRunning() is true on the // SpatialMappingManager.Instance. if(SpatialMappingManager.Instance.IsObserverRunning()) { // 3.a: If running, Stop the observer by calling // StopObserver() on the SpatialMappingManager.Instance. SpatialMappingManager.Instance.StopObserver(); } // 3.a: Call CreatePlanes() to generate planes. CreatePlanes(); // 3.a: Set meshesProcessed to true. meshesProcessed = true; } } } /// <summary> /// Handler for the SurfaceMeshesToPlanes MakePlanesComplete event. /// </summary> /// <param name="source">Source of the event.</param> /// <param name="args">Args for the event.</param> private void SurfaceMeshesToPlanes_MakePlanesComplete(object source, System.EventArgs args) { /* TODO: 3.a DEVELOPER CODING EXERCISE 3.a */ // Collection of floor and table planes that we can use to set horizontal items on. List<GameObject> horizontal = new List<GameObject>(); // Collection of wall planes that we can use to set vertical items on. List<GameObject> vertical = new List<GameObject>(); // 3.a: Get all floor and table planes by calling // SurfaceMeshesToPlanes.Instance.GetActivePlanes(). // Assign the result to the ‘horizontal‘ list. horizontal = SurfaceMeshesToPlanes.Instance.GetActivePlanes(PlaneTypes.Table | PlaneTypes.Floor); // 3.a: Get all wall planes by calling // SurfaceMeshesToPlanes.Instance.GetActivePlanes(). // Assign the result to the ‘vertical‘ list. vertical = SurfaceMeshesToPlanes.Instance.GetActivePlanes(PlaneTypes.Wall); // Check to see if we have enough horizontal planes (minimumFloors) // and vertical planes (minimumWalls), to set holograms on in the world. if (horizontal.Count >= minimumFloors && vertical.Count >= minimumWalls) { // We have enough floors and walls to place our holograms on... // 3.a: Let‘s reduce our triangle count by removing triangles // from SpatialMapping meshes that intersect with our active planes. // Call RemoveVertices(). // Pass in all activePlanes found by SurfaceMeshesToPlanes.Instance. RemoveVertices(SurfaceMeshesToPlanes.Instance.ActivePlanes); // 3.a: We can indicate to the user that scanning is over by // changing the material applied to the Spatial Mapping meshes. // Call SpatialMappingManager.Instance.SetSurfaceMaterial(). // Pass in the secondaryMaterial. SpatialMappingManager.Instance.SetSurfaceMaterial(secondaryMaterial); // 3.a: We are all done processing the mesh, so we can now // initialize a collection of Placeable holograms in the world // and use horizontal/vertical planes to set their starting positions. // Call SpaceCollectionManager.Instance.GenerateItemsInWorld(). // Pass in the lists of horizontal and vertical planes that we found earlier. SpaceCollectionManager.Instance.GenerateItemsInWorld(horizontal, vertical); } else { // We do not have enough floors/walls to place our holograms on... // 3.a: Re-enter scanning mode so the user can find more surfaces by // calling StartObserver() on the SpatialMappingManager.Instance. SpatialMappingManager.Instance.StartObserver(); // 3.a: Re-process spatial data after scanning completes by // re-setting meshesProcessed to false. meshesProcessed = false; } } /// <summary> /// Creates planes from the spatial mapping surfaces. /// </summary> private void CreatePlanes() { // Generate planes based on the spatial map. SurfaceMeshesToPlanes surfaceToPlanes = SurfaceMeshesToPlanes.Instance; if (surfaceToPlanes != null && surfaceToPlanes.enabled) { surfaceToPlanes.MakePlanes(); } } /// <summary> /// Removes triangles from the spatial mapping surfaces. /// </summary> /// <param name="boundingObjects"></param> private void RemoveVertices(IEnumerable<GameObject> boundingObjects) { RemoveSurfaceVertices removeVerts = RemoveSurfaceVertices.Instance; if (removeVerts != null && removeVerts.enabled) { removeVerts.RemoveSurfaceVerticesWithinBounds(boundingObjects); } } /// <summary> /// Called when the GameObject is unloaded. /// </summary> private void OnDestroy() { if (SurfaceMeshesToPlanes.Instance != null) { SurfaceMeshesToPlanes.Instance.MakePlanesComplete -= SurfaceMeshesToPlanes_MakePlanesComplete; } } }
部署发布:
- 在部署到HoloLens之前,按Unity中的播放按钮进入播放模式。
- 在文件加载房间网格后,等待10秒钟,这时在对空间映射网格进行处理。
- 当处理完成时,地板,墙壁,天花板等将以平面版的形式呈现。
- 在找到所有的平面之后,你应该会看到一个太阳系,在靠近相机的地板上。
- 两个海报应该出现在相机附近的墙上。 如果在游戏模式下看不到它们,请切换到场景选项卡。
- 再次按下播放按钮退出播放模式。
- 像往常一样构建和部署该项目到HoloLens。
- 等待扫描和处理空间映射数据完成。
- 一旦你看到场景中平面生成,试着找到你的世界中的太阳系和海报。
章节4 放置
这章主要学习实现如何将你的全息对象通过手势操作将它放置在你想放置的平面上,如果你想放置的位置不具备放置的条件,提供视觉反馈提醒用户不能放置。
步骤:
- 在Unity的 Hierarchy 面板选择 SpatialProcessing 对象.
- 在 Inspector 面板中找到 Surface Meshes To Planes (Script) 组件.
- 改变 Draw Planes 属性为 Nothing .
- 改变 Draw Planes 属性 Wall, 这时只有樯面会被渲染成平面状态.
- 在 Project 面板下, Scripts 文件夹, 双击 Placeable.cs 脚本用VS打开.
这个Placeable 脚本已附加海报和投影框上。 我们需要做的是取消注释一些代码,这个脚本将实现以下:
1.通过从立方体的中心和边界四个角进行光线投射,确定全息图是否匹配到了一个表面上。
2.检查表面法线以确定它是否足够平滑以使全息图齐平。
3.渲染出一个围绕全息图的立方体边框,以显示全息图放置时的实际尺寸。
4.在全息图下面/后面投一个阴影,以显示它将放在地板/墙上的位置。
5.如果全息图不能放置在表面上,渲染阴影为红色,如果可以放置,则将阴影渲染为绿色。
6.重新定向全息图以与具有亲和力的表面类型(垂直或水平)对齐。
7.平滑稳定地将全息图放置在所选表面上,以避免跳跃或断裂行为。
using System.Collections.Generic; using UnityEngine; using HoloToolkit.Unity; /// <summary> /// Enumeration containing the surfaces on which a GameObject /// can be placed. For simplicity of this sample, only one /// surface type is allowed to be selected. /// </summary> public enum PlacementSurfaces { // Horizontal surface with an upward pointing normal. Horizontal = 1, // Vertical surface with a normal facing the user. Vertical = 2, } /// <summary> /// The Placeable class implements the logic used to determine if a GameObject /// can be placed on a target surface. Constraints for placement include: /// * No part of the GameObject‘s box collider impacts with another object in the scene /// * The object lays flat (within specified tolerances) against the surface /// * The object would not fall off of the surface if gravity were enabled. /// This class also provides the following visualizations. /// * A transparent cube representing the object‘s box collider. /// * Shadow on the target surface indicating whether or not placement is valid. /// </summary> public class Placeable : MonoBehaviour { [Tooltip("The base material used to render the bounds asset when placement is allowed.")] public Material PlaceableBoundsMaterial = null; [Tooltip("The base material used to render the bounds asset when placement is not allowed.")] public Material NotPlaceableBoundsMaterial = null; [Tooltip("The material used to render the placement shadow when placement it allowed.")] public Material PlaceableShadowMaterial = null; [Tooltip("The material used to render the placement shadow when placement it not allowed.")] public Material NotPlaceableShadowMaterial = null; [Tooltip("The type of surface on which the object can be placed.")] public PlacementSurfaces PlacementSurface = PlacementSurfaces.Horizontal; [Tooltip("The child object(s) to hide during placement.")] public List<GameObject> ChildrenToHide = new List<GameObject>(); /// <summary> /// Indicates if the object is in the process of being placed. /// </summary> public bool IsPlacing { get; private set; } // The most recent distance to the surface. This is used to // locate the object when the user‘s gaze does not intersect // with the Spatial Mapping mesh. private float lastDistance = 2.0f; // The distance away from the target surface that the object should hover prior while being placed. private float hoverDistance = 0.15f; // Threshold (the closer to 0, the stricter the standard) used to determine if a surface is flat. private float distanceThreshold = 0.02f; // Threshold (the closer to 1, the stricter the standard) used to determine if a surface is vertical. private float upNormalThreshold = 0.9f; // Maximum distance, from the object, that placement is allowed. // This is used when raycasting to see if the object is near a placeable surface. private float maximumPlacementDistance = 5.0f; // Speed (1.0 being fastest) at which the object settles to the surface upon placement. private float placementVelocity = 0.06f; // Indicates whether or not this script manages the object‘s box collider. private bool managingBoxCollider = false; // The box collider used to determine of the object will fit in the desired location. // It is also used to size the bounding cube. private BoxCollider boxCollider = null; // Visible asset used to show the dimensions of the object. This asset is sized // using the box collider‘s bounds. private GameObject boundsAsset = null; // Visible asset used to show the where the object is attempting to be placed. // This asset is sized using the box collider‘s bounds. private GameObject shadowAsset = null; // The location at which the object will be placed. private Vector3 targetPosition; /// <summary> /// Called when the GameObject is created. /// </summary> private void Awake() { targetPosition = gameObject.transform.position; // Get the object‘s collider. boxCollider = gameObject.GetComponent<BoxCollider>(); if (boxCollider == null) { // The object does not have a collider, create one and remember that // we are managing it. managingBoxCollider = true; boxCollider = gameObject.AddComponent<BoxCollider>(); boxCollider.enabled = false; } // Create the object that will be used to indicate the bounds of the GameObject. boundsAsset = GameObject.CreatePrimitive(PrimitiveType.Cube); boundsAsset.transform.parent = gameObject.transform; boundsAsset.SetActive(false); // Create a object that will be used as a shadow. shadowAsset = GameObject.CreatePrimitive(PrimitiveType.Quad); shadowAsset.transform.parent = gameObject.transform; shadowAsset.SetActive(false); } /// <summary> /// Called when our object is selected. Generally called by /// a gesture management component. /// </summary> public void OnSelect() { /* TODO: 4.a CODE ALONG 4.a */ if (!IsPlacing) { OnPlacementStart(); } else { OnPlacementStop(); } } /// <summary> /// Called once per frame. /// </summary> private void Update() { /* TODO: 4.a CODE ALONG 4.a */ if (IsPlacing) { // Move the object. Move(); // Set the visual elements. Vector3 targetPosition; Vector3 surfaceNormal; bool canBePlaced = ValidatePlacement(out targetPosition, out surfaceNormal); DisplayBounds(canBePlaced); DisplayShadow(targetPosition, surfaceNormal, canBePlaced); } else { // Disable the visual elements. boundsAsset.SetActive(false); shadowAsset.SetActive(false); // Gracefully place the object on the target surface. float dist = (gameObject.transform.position - targetPosition).magnitude; if (dist > 0) { gameObject.transform.position = Vector3.Lerp(gameObject.transform.position, targetPosition, placementVelocity / dist); } else { // Unhide the child object(s) to make placement easier. for (int i = 0; i < ChildrenToHide.Count; i++) { ChildrenToHide[i].SetActive(true); } } } } /// <summary> /// Verify whether or not the object can be placed. /// </summary> /// <param name="position"> /// The target position on the surface. /// </param> /// <param name="surfaceNormal"> /// The normal of the surface on which the object is to be placed. /// </param> /// <returns> /// True if the target position is valid for placing the object, otherwise false. /// </returns> private bool ValidatePlacement(out Vector3 position, out Vector3 surfaceNormal) { Vector3 raycastDirection = gameObject.transform.forward; if (PlacementSurface == PlacementSurfaces.Horizontal) { // Placing on horizontal surfaces. // Raycast from the bottom face of the box collider. raycastDirection = -(Vector3.up); } // Initialize out parameters. position = Vector3.zero; surfaceNormal = Vector3.zero; Vector3[] facePoints = GetColliderFacePoints(); // The origin points we receive are in local space and we // need to raycast in world space. for (int i = 0; i < facePoints.Length; i++) { facePoints[i] = gameObject.transform.TransformVector(facePoints[i]) + gameObject.transform.position; } // Cast a ray from the center of the box collider face to the surface. RaycastHit centerHit; if (!Physics.Raycast(facePoints[0], raycastDirection, out centerHit, maximumPlacementDistance, SpatialMappingManager.Instance.LayerMask)) { // If the ray failed to hit the surface, we are done. return false; } // We have found a surface. Set position and surfaceNormal. position = centerHit.point; surfaceNormal = centerHit.normal; // Cast a ray from the corners of the box collider face to the surface. for (int i = 1; i < facePoints.Length; i++) { RaycastHit hitInfo; if (Physics.Raycast(facePoints[i], raycastDirection, out hitInfo, maximumPlacementDistance, SpatialMappingManager.Instance.LayerMask)) { // To be a valid placement location, each of the corners must have a similar // enough distance to the surface as the center point if (!IsEquivalentDistance(centerHit.distance, hitInfo.distance)) { return false; } } else { // The raycast failed to intersect with the target layer. return false; } } return true; } /// <summary> /// Determine the coordinates, in local space, of the box collider face that /// will be placed against the target surface. /// </summary> /// <returns> /// Vector3 array with the center point of the face at index 0. /// </returns> private Vector3[] GetColliderFacePoints() { // Get the collider extents. // The size values are twice the extents. Vector3 extents = boxCollider.size / 2; // Calculate the min and max values for each coordinate. float minX = boxCollider.center.x - extents.x; float maxX = boxCollider.center.x + extents.x; float minY = boxCollider.center.y - extents.y; float maxY = boxCollider.center.y + extents.y; float minZ = boxCollider.center.z - extents.z; float maxZ = boxCollider.center.z + extents.z; Vector3 center; Vector3 corner0; Vector3 corner1; Vector3 corner2; Vector3 corner3; if (PlacementSurface == PlacementSurfaces.Horizontal) { // Placing on horizontal surfaces. center = new Vector3(boxCollider.center.x, minY, boxCollider.center.z); corner0 = new Vector3(minX, minY, minZ); corner1 = new Vector3(minX, minY, maxZ); corner2 = new Vector3(maxX, minY, minZ); corner3 = new Vector3(maxX, minY, maxZ); } else { // Placing on vertical surfaces. center = new Vector3(boxCollider.center.x, boxCollider.center.y, maxZ); corner0 = new Vector3(minX, minY, maxZ); corner1 = new Vector3(minX, maxY, maxZ); corner2 = new Vector3(maxX, minY, maxZ); corner3 = new Vector3(maxX, maxY, maxZ); } return new Vector3[] { center, corner0, corner1, corner2, corner3 }; } /// <summary> /// Put the object into placement mode. /// </summary> public void OnPlacementStart() { // If we are managing the collider, enable it. if (managingBoxCollider) { boxCollider.enabled = true; } // Hide the child object(s) to make placement easier. for (int i = 0; i < ChildrenToHide.Count; i++) { ChildrenToHide[i].SetActive(false); } // Tell the gesture manager that it is to assume // all input is to be given to this object. GestureManager.Instance.OverrideFocusedObject = gameObject; // Enter placement mode. IsPlacing = true; } /// <summary> /// Take the object out of placement mode. /// </summary> /// <remarks> /// This method will leave the object in placement mode if called while /// the object is in an invalid location. To determine whether or not /// the object has been placed, check the value of the IsPlacing property. /// </remarks> public void OnPlacementStop() { // ValidatePlacement requires a normal as an out parameter. Vector3 position; Vector3 surfaceNormal; // Check to see if we can exit placement mode. if (!ValidatePlacement(out position, out surfaceNormal)) { return; } // The object is allowed to be placed. // We are placing at a small buffer away from the surface. targetPosition = position + (0.01f * surfaceNormal); OrientObject(true, surfaceNormal); // If we are managing the collider, disable it. if (managingBoxCollider) { boxCollider.enabled = false; } // Tell the gesture manager that it is to resume // its normal behavior. GestureManager.Instance.OverrideFocusedObject = null; // Exit placement mode. IsPlacing = false; } /// <summary> /// Positions the object along the surface toward which the user is gazing. /// </summary> /// <remarks> /// If the user‘s gaze does not intersect with a surface, the object /// will remain at the most recently calculated distance. /// </remarks> private void Move() { Vector3 moveTo = gameObject.transform.position; Vector3 surfaceNormal = Vector3.zero; RaycastHit hitInfo; bool hit = Physics.Raycast(Camera.main.transform.position, Camera.main.transform.forward, out hitInfo, 20f, SpatialMappingManager.Instance.LayerMask); if (hit) { float offsetDistance = hoverDistance; // Place the object a small distance away from the surface while keeping // the object from going behind the user. if (hitInfo.distance <= hoverDistance) { offsetDistance = 0f; } moveTo = hitInfo.point + (offsetDistance * hitInfo.normal); lastDistance = hitInfo.distance; surfaceNormal = hitInfo.normal; } else { // The raycast failed to hit a surface. In this case, keep the object at the distance of the last // intersected surface. moveTo = Camera.main.transform.position + (Camera.main.transform.forward * lastDistance); } // Follow the user‘s gaze. float dist = Mathf.Abs((gameObject.transform.position - moveTo).magnitude); gameObject.transform.position = Vector3.Lerp(gameObject.transform.position, moveTo, placementVelocity / dist); // Orient the object. // We are using the return value from Physics.Raycast to instruct // the OrientObject function to align to the vertical surface if appropriate. OrientObject(hit, surfaceNormal); } /// <summary> /// Orients the object so that it faces the user. /// </summary> /// <param name="alignToVerticalSurface"> /// If true and the object is to be placed on a vertical surface, /// orient parallel to the target surface. If false, orient the object /// to face the user. /// </param> /// <param name="surfaceNormal"> /// The target surface‘s normal vector. /// </param> /// <remarks> /// The aligntoVerticalSurface parameter is ignored if the object /// is to be placed on a horizontalSurface /// </remarks> private void OrientObject(bool alignToVerticalSurface, Vector3 surfaceNormal) { Quaternion rotation = Camera.main.transform.localRotation; // If the user‘s gaze does not intersect with the Spatial Mapping mesh, // orient the object towards the user. if (alignToVerticalSurface && (PlacementSurface == PlacementSurfaces.Vertical)) { // We are placing on a vertical surface. // If the normal of the Spatial Mapping mesh indicates that the // surface is vertical, orient parallel to the surface. if (Mathf.Abs(surfaceNormal.y) <= (1 - upNormalThreshold)) { rotation = Quaternion.LookRotation(-surfaceNormal, Vector3.up); } } else { rotation.x = 0f; rotation.z = 0f; } gameObject.transform.rotation = rotation; } /// <summary> /// Displays the bounds asset. /// </summary> /// <param name="canBePlaced"> /// Specifies if the object is in a valid placement location. /// </param> private void DisplayBounds(bool canBePlaced) { // Ensure the bounds asset is sized and positioned correctly. boundsAsset.transform.localPosition = boxCollider.center; boundsAsset.transform.localScale = boxCollider.size; boundsAsset.transform.rotation = gameObject.transform.rotation; // Apply the appropriate material. if (canBePlaced) { boundsAsset.GetComponent<Renderer>().sharedMaterial = PlaceableBoundsMaterial; } else { boundsAsset.GetComponent<Renderer>().sharedMaterial = NotPlaceableBoundsMaterial; } // Show the bounds asset. boundsAsset.SetActive(true); } /// <summary> /// Displays the placement shadow asset. /// </summary> /// <param name="position"> /// The position at which to place the shadow asset. /// </param> /// <param name="surfaceNormal"> /// The normal of the surface on which the asset will be placed /// </param> /// <param name="canBePlaced"> /// Specifies if the object is in a valid placement location. /// </param> private void DisplayShadow(Vector3 position, Vector3 surfaceNormal, bool canBePlaced) { // Rotate the shadow so that it is displayed on the correct surface and matches the object. float rotationX = 0.0f; if (PlacementSurface == PlacementSurfaces.Horizontal) { rotationX = 90.0f; } Quaternion rotation = Quaternion.Euler(rotationX, gameObject.transform.rotation.eulerAngles.y, 0); shadowAsset.transform.localScale = boxCollider.size; shadowAsset.transform.rotation = rotation; // Apply the appropriate material. if (canBePlaced) { shadowAsset.GetComponent<Renderer>().sharedMaterial = PlaceableShadowMaterial; } else { shadowAsset.GetComponent<Renderer>().sharedMaterial = NotPlaceableShadowMaterial; } // Show the shadow asset as appropriate. if (position != Vector3.zero) { // Position the shadow a small distance from the target surface, along the normal. shadowAsset.transform.position = position + (0.01f * surfaceNormal); shadowAsset.SetActive(true); } else { shadowAsset.SetActive(false); } } /// <summary> /// Determines if two distance values should be considered equivalent. /// </summary> /// <param name="d1"> /// Distance to compare. /// </param> /// <param name="d2"> /// Distance to compare. /// </param> /// <returns> /// True if the distances are within the desired tolerance, otherwise false. /// </returns> private bool IsEquivalentDistance(float d1, float d2) { float dist = Mathf.Abs(d1 - d2); return (dist <= distanceThreshold); } /// <summary> /// Called when the GameObject is unloaded. /// </summary> private void OnDestroy() { // Unload objects we have created. Destroy(boundsAsset); boundsAsset = null; Destroy(shadowAsset); shadowAsset = null; } }
发布测试:
- 和前面一样,构建项目并部署到HoloLens。
- 等待扫描和空间映射数据预处理完成。
- 当你看到太阳系时,注视下面的投影框,并执行选择手势移动它。 当投影框被选中时,在投影框周围将出现一个边界立方体。
- 移动你的头,凝视房间里其他的位置。 投影盒应该会遵循你的目光。 当投影框下方的阴影变为红色时,表示不能将全息图放置在该表面上。 当投影框下方的阴影变为绿色时,您可以通过执行另一个选择手势将它放置在这个位置。
- 找到一个你房间墙上的全息海报并选择它,将其移动到一个新的位置。 请注意,您不能将海报放置在地板或天花板上,并且在您四处移动时,海报会保持正确朝向(即始终保持与墙壁垂直,并面向室内)。
章节5 遮挡
这章主要实现判断全息对象是否被空间映射的网格所遮挡,如果被遮挡,赋予该全息对象X-射线的视觉效果以表明它被遮挡。
步骤:
首先,我们将使得空间映射网格遮挡其他全息图,而不遮挡真实世界:
- 在 Hierarchy 面板下选择SpatialProcessing 对象.
- 在 Inspector 面板中找到 Play Space Manager (Script) 组件.
- 点击Secondary Material 属性右侧的小圆圈.
- 在弹出的窗口中搜索 Occlusion 材质球并选中,然后关闭此窗口.
接下来,我们将给地球添加一个特殊的效果,当它在被另一个全息图(如太阳)遮挡或者被空间映射网格所遮挡时具有蓝色高光:
- 在 Project 面板, Holograms 文件夹, 展开 SolarSystem 对象.
- 点击 Earth.
- 在 Inspector 面板中, 找到地球的材质球 .
- 将其shader 改选为 Custom > OcclusionRim.当地球被另一个物体遮挡时,这将在地球上呈现一个蓝色的高光。
最后,我们将为我们的太阳系中的行星实现X射线的视觉效果。 我们将需要编辑PlanetOcclusion.cs(在Scripts \ SolarSystem文件夹中可以找到),以便实现以下操作:
1.确定行星是否被SpatialMapping层(房间网格和平面)遮挡。
2.只要行星被SpatialMapping图层遮挡,显示它的线框效果。
3.当行星没有被SpatialMapping图层阻挡时,隐藏它的线框效果。
using UnityEngine; using HoloToolkit.Unity; /// <summary> /// Determines when the occluded version of the planet should be visible. /// This script allows us to do selective occlusion, so the occlusionObject /// will only be rendered when a Spatial Mapping surface is occluding the planet, /// not when another hologram is responsible for the occlusion. /// </summary> public class PlanetOcclusion : MonoBehaviour { [Tooltip("Object to display when the planet is occluded.")] public GameObject occlusionObject; /// <summary> /// Points to raycast to when checking for occlusion. /// </summary> private Vector3[] checkPoints; // Use this for initialization void Start() { occlusionObject.SetActive(false); // Set the check points to use when testing for occlusion. MeshFilter filter = gameObject.GetComponent<MeshFilter>(); Vector3 extents = filter.mesh.bounds.extents; Vector3 center = filter.mesh.bounds.center; Vector3 top = new Vector3(center.x, center.y + extents.y, center.z); Vector3 left = new Vector3(center.x - extents.x, center.y, center.z); Vector3 right = new Vector3(center.x + extents.x, center.y, center.z); Vector3 bottom = new Vector3(center.x, center.y - extents.y, center.z); checkPoints = new Vector3[] { center, top, left, right, bottom }; } // Update is called once per frame void Update() { /* TODO: 5.a DEVELOPER CODING EXERCISE 5.a */ // Check to see if any of the planet‘s boundary points are occluded. for (int i = 0; i < checkPoints.Length; i++) { // 5.a: Convert the current checkPoint to world coordinates. // Call gameObject.transform.TransformPoint(checkPoints[i]). // Assign the result to a new Vector3 variable called ‘checkPt‘. Vector3 checkPt = gameObject.transform.TransformPoint(checkPoints[i]); // 5.a: Call Vector3.Distance() to calculate the distance // between the Main Camera‘s position and ‘checkPt‘. // Assign the result to a new float variable called ‘distance‘. float distance = Vector3.Distance(Camera.main.transform.position, checkPt); // 5.a: Take ‘checkPt‘ and subtract the Main Camera‘s position from it. // Assign the result to a new Vector3 variable called ‘direction‘. Vector3 direction = checkPt - Camera.main.transform.position; // Used to indicate if the call to Physics.Raycast() was successful. bool raycastHit = false; // 5.a: Check if the planet is occluded by a spatial mapping surface. // Call Physics.Raycast() with the following arguments: // - Pass in the Main Camera‘s position as the origin. // - Pass in ‘direction‘ for the direction. // - Pass in ‘distance‘ for the maxDistance. // - Pass in SpatialMappingManager.Instance.LayerMask as layerMask. // Assign the result to ‘raycastHit‘. raycastHit = Physics.Raycast(Camera.main.transform.position, direction, distance, SpatialMappingManager.Instance.LayerMask); if (raycastHit) { // 5.a: Our raycast hit a surface, so the planet is occluded. // Set the occlusionObject to active. occlusionObject.SetActive(true); // At least one point is occluded, so break from the loop. break; } else { // 5.a: The Raycast did not hit, so the planet is not occluded. // Deactivate the occlusionObject. occlusionObject.SetActive(false); } } } }
部署发布:
- 像往常一样,将应用程序构建并部署到HoloLens。
- 等待空间映射数据的扫描和处理完成(您应该看到墙上出现蓝线)。
- 找到并选择太阳系的投影框,然后将它放置在墙壁旁边或在柜台后面。
- 您可以通过将海报或投影框隐藏在其他对象的后面,以查看基本的遮挡效果。
- 观察地球,每当它在另一个全息图或表面后面时,应该有一个蓝色的高光效果。
- 观察行星移动到墙壁或房间里的其他表面后面时。 你会看到X光射线的效果,可以看到他们的线框骨架!
原文链接:https://developer.microsoft.com/en-us/windows/holographic/holograms_230
本人按照自己的理解对原文稍作修改,如有不恰当的地方,请指正哦!
微软Hololens学院教程-Hologram 230-空间映射(Spatial mapping )