基于视觉的自动驾驶环境感知技术解析

智驾最前沿 2023-04-28 08:45

TI MCU方案：电动汽车实时控制 如何增强电动汽车的实时控制能力？

--关注、星标、回复“SOA”--

↓↓领取：面向智能车辆开发的开放性SOA方案↓↓

视觉感知在自动驾驶中起着重要作用，主要任务之一是目标检测和识别。由于视觉传感器具有丰富的颜色和纹理信息，因此可以快速准确地识别各种道路信息。常用的技术基于提取和计算图像的各种特征，基于深度学习的方法具有更好的可靠性和处理速度，并且在识别复杂元素方面具有更大的优势。对于深度估计，视觉传感器由于其尺寸小和成本低，也被用于测距。单目相机使用来自单个视点的图像数据作为输入来估计对象深度，相比之下，立体视觉是基于视差和匹配不同视图的特征点，深度学习的应用也进一步提高了准确性。

此外，SLAM可以建立道路环境模型，从而帮助车辆感知周围环境并完成任务。本文介绍并比较了各种目标检测和识别方法，然后解释了深度估计的发展，并比较了基于单目、立体和RGB-D传感器的各种方法，接下来回顾并比较了SLAM的各种方法。最后总结了当前存在的问题，并提出了视觉技术的未来发展趋势。

应用背景与状况

环境感知是自动驾驶最重要的功能之一，环境感知的性能，如准确性、对光变化和阴影噪声的鲁棒性，以及对复杂道路环境和恶劣天气的适应性，直接影响自动驾驶技术的性能。自动驾驶中常用的传感器包括超声波雷达、毫米波雷达、激光雷达、视觉传感器等。尽管全球定位技术（如GPS、北斗、GLONASS等）相对成熟，能够进行全天候定位，但仍存在信号阻塞甚至丢失、更新频率低，以及在诸如城市建筑物和隧道的环境中的定位精度差的问题。

里程计定位具有更新频率快、短期精度高的优点，但长期累积误差较大。尽管激光雷达具有高精度，但也存在一些缺点，例如体积大、成本高和依赖天气。特斯拉和几家公司，如Mobileye、Apollo和MAXIEYE，使用视觉传感器进行环境感知。视觉传感器在自动驾驶中的应用有助于物体检测和图像处理，以分析障碍物和可驾驶区域，从而确保车辆安全到达目的地。与其它传感器相比，视觉图像尤其是彩色图像信息量极大，它们不仅包含物体的距离信息，还包含颜色、纹理和深度信息，从而能够通过信号检测同时进行车道线检测、车辆检测、行人检测、交通标志检测等。此外，不同车辆上的摄像头之间没有干扰。视觉传感器还可以实现同时定位和地图构建（SLAM）。

视觉环境感知在自动驾驶中的主要应用是目标检测识别、深度估计和SLAM。根据相机的工作原理，视觉传感器可以分为三大类：单目、立体和RGB-D。单目相机只有一个相机，立体相机有多个相机。RGB-D更复杂，除了能够捕捉彩色图像之外，它还搭载了几个不同的相机，可以读取每个像素与相机之间的距离。此外，视觉传感器与机器学习、深度学习和其它人工智能的集成可以获得更好的检测结果，本文将讨论以下三个方面。

1）基于视觉的物体检测和识别，包括传统方法和基于深度学习的方法；

2）基于单目、立体和RGBD的深度估计以及深度学习的应用；

3）单目SLAM、立体SLAM和RGBD SLAM；

目标检测与识别

1）传统方法

在自动驾驶中，识别道路、车辆和行人等道路要素，然后做出不同的决定是车辆安全驾驶的基础。目标检测和识别的工作流程如图1所示。图像采集由拍摄车身周围环境照片的相机进行，特斯拉使用了广角、中等焦距和长焦相机的组合。广角相机的视角约为150°，负责识别附近区域的大范围物体。中等焦距相机的视角约为50°，负责识别车道线、车辆、行人、红绿灯和其他信息。长焦相机的视角只有35°左右，但识别距离可以达到200~250m。它用于识别远处的行人、车辆、路标和其他信息，并通过多个相机的组合更全面地收集道路信息。

图像预处理消除了图像中的无关信息，保留了有用信息，增强了相关信息的可检测性，并简化了数据，从而提高了特征提取、图像分割、匹配和识别的可靠性。

为了完成图像中目标的识别，需要提取所需的特征，实现精确匹配，主要包括边缘特征、外观特征（轮廓、纹理、分散度和拓扑特征）、统计特征（如均值、方差、能量、熵等）和其它特征等。识别算法则是借助于匹配和机器学习方案！

2）基于深度学习的方法

与传统的目标检测和识别相比，深度学习需要基于大数据集的训练，但会带来更好的性能。传统的目标识别方法分别进行特征提取和分类器设计，然后将它们结合在一起。相比之下，深度学习具有更强大的特征学习和特征表示能力，通过学习数据库和映射关系，将相机捕获的信息处理到向量空间中，以便通过神经网络进行识别。

one-stage目标检测算法包括YOLO系列、SSD、Nanodet、YOLOX这类算法，two-stage算法主要包括Faster RCNN、Cascade RCNN范式。

深度估计

在自动驾驶系统中，适当的距离对于确保汽车的安全驾驶非常重要，因此需要从图像中进行深度估计。深度估计的目标是获得到物体的距离，并最终获得深度图，该深度图为一系列任务（如3D重建、SLAM和决策）提供深度信息，目前市场上主流的距离测量方法是单目、立体和基于RGBD相机的。

1）传统深度估计方法

对于固定的单目相机和物体，由于无法直接测量深度信息，因此，单目深度估计是先识别，然后测量距离。首先，通过图像匹配进行识别，然后根据数据库中目标的大小进行距离估计。由于在识别和估计阶段都需要与已建立的样本数据库进行比较，因此它缺乏自学习功能，并且感知结果受到数据库的限制，并且通常忽略未标记的目标，这导致了无法识别不常见目标的问题。然而，对于应用于自动驾驶的单目深度估计，目标主要是已知目标，例如车辆和行人，因此可以使用几何关系方法、数据回归建模方法和逆透视映射，并且可以通过车辆的运动来实现基于SFM（来自运动的结构）的单目高度估计。目前，单目相机由于其成本低、检测速度快、能够识别特定障碍物类型、算法成熟度高和识别准确，正逐渐成为视觉测距的主流技术。

几何关系法使用针孔相机成像原理。它使用沿着直线的光传播将三维世界中的物体投射到二维成像平面上，如图9所示，车辆距离可以通过图中的等式来计算。然而，要求摄像机的光轴必须与水平地面平行，这在实践中很难保证。

数据回归建模方法通过拟合函数来测量距离，以获得像素距离和实际距离之间的非线性关系。逆透视映射不仅广泛应用于单目测距，也广泛应用于环视相机。通过将透视图转换为“鸟瞰图”，如图10所示。由于“鸟瞰图”与真实道路平面具有线性比例关系，因此通过校准比例因子，可以根据逆透视变换视图中的像素距离计算实际车辆距离，这简单且易于实现。

然而，没有考虑汽车的俯仰和偏航运动，并且俯仰角的存在将使反向透视变换俯视图无法恢复实际道路俯视图的平行度，从而产生较大的测距误差。[91]提出了一种基于可变参数逆透视变换的距离测量模型，该模型动态补偿摄像机的俯仰角，不同道路环境下的车辆测距误差在5%以内，实时鲁棒性高。然而，在没有车道线和清晰道路边界的非结构化道路上，无法计算摄像机的俯仰角。[92]中提出了一种无累积误差的俯仰角估计方法，该方法使用Harris角点算法和金字塔Lucas Kanade方法来检测相机相邻帧之间的特征点。通过特征点匹配和成对几何约束求解其相机旋转矩阵和平移向量，并使用高斯-牛顿方法进行参数优化。然后，从旋转矩阵分解俯仰角速率，并从平移向量计算俯仰角。

SFM（Structure From Motion）是通过使用诸如多视图几何优化之类的数学理论从2D图像序列中确定目标的空间几何关系，以通过相机移动恢复3D结构。SFM方便灵活，但在图像序列采集中遇到场景和运动退化问题。根据图像添加顺序的拓扑结构，可以将其分为增量/顺序SFM、全局SFM、混合SFM和分层SFM。此外，还有语义SFM和基于深度学习的SFM。

另一方面，混合SFM结合了增量SFM和全局SFM的优点，并逐渐成为一种趋势。该pipelines可以概括为摄像机旋转矩阵的全局估计、摄像机中心的增量计算以及针对全局敏感问题的基于社区的旋转平均方法。与混合SFM相比，PSFM[109]将相机分为多个集群，在大规模场景和高精度重建方面具有优势。[110]建议SFMLearner使用光度一致性原理来估计每个帧的深度和姿态。基于此，[111]提出了SFM网络，添加了光流、场景流和3D点云来估计深度。

单目摄像头具有较高的接近识别率，因此被广泛应用于正面碰撞预警系统（FCWS），但它的环境适应性较差，在车辆行驶时，摄像头会因颠簸而抖动。在[113]中，对三种场景（静止、慢速移动和制动）进行了比较实验，结果将TTC的算术平均值作为警报阈值，这可以有效地避免摄像机抖动等异常情况，因此可以应用于更复杂的范围。[114]采用了消失点检测、车道线提取和3D空间车辆检测的组合来实现距离测量。然而，在照明不足和前方严重障碍物遮挡的情况下，距离误差显著增加。在[115]中，提出在检测和测距车辆前方的物体之前，使用单目视觉里程表结合GPS路面特征和几何测量来估计系统的绝对比例和姿态，并且可以使用物体的3D形状变化来实现相机本身和物体的定位。

2）基于深度学习的方式

基于深度学习的单目深度估计的输入是捕获的原始图像，并且输出是深度图，其中每个像素值对应于输入图像的场景深度。基于深度学习的单目深度估计算法分为有监督学习和无监督学习，监督学习能够高精度地从单个图像和场景的结构中恢复尺度信息，因为它们使用地面真实深度值直接训练网络，但需要KITTI、Open Image、Kinetics、JFT-300M等数据集。

[116]使用马尔可夫随机场（MRF）来学习输入图像特征和输出深度之间的映射关系，但RGB图像和深度之间的关系需要人为假设。该模型难以模拟真实世界的映射关系，因此预测精度有限。2014年，[117]提出在多层中卷积和下采样图像，以获得整个场景的描述性特征，并使用它们来预测全局深度。然后，预测图像的局部信息由第二分支网络细化，其中全局深度将被用作局部分支的输入，以帮助预测局部深度。2015年，[118]基于上述工作提出了统一的多尺度网络框架。该框架使用更深的基础网络VGG，并使用第三精细尺度网络来进一步添加详细信息，以提高分辨率，从而更好地进行深度估计。2016年，[119]使用卷积神经网络模型进行车辆检测和定位，然后基于单目视觉原理计算距离。2018年，[120]提出了DORN框架，将连续深度值划分为离散区间，然后使用完全连接的层对卷积进行解码和膨胀，以进行特征提取和距离测量。同年，[121]比较了激光雷达，将输入图像转换为与激光雷达生成的点云数据相似的点云，然后使用点云和图像融合算法来检测和测量距离。[122]提议的MonoGRNet，它通过ROIAlign获取对象的视觉特征，然后使用这些特征来预测对象的3D中心的深度。2019年，[123]通过提出MonoGRNetV2将centROId扩展到多个关键点并使用3D CAD对象模型进行深度估计，对其进行了改进。[124]提出了BEV-IPM将图像从透视图转换为鸟瞰图（BEV）。在BEV视图中，基于YOLO网络检测底框（物体和路面之间的接触部分），然后使用神经网络预测的Box精确估计其距离。[125]建议使用卷积神经网络输出的多尺度特征图，基于两个分辨率的深度估计来预测不同分辨率的深度图，并且通过连续的MRF融合不同分辨率的特征图，以获得与输入图像相对应的深度图。[126]提出了3D-RCNN，其首先使用PCA缩小参数空间，然后基于R-CNN预测的每个目标低维模型参数生成2D图像和深度图。然而，只有在较低的空间分辨率下，CNN才能更好地处理全局信息。单目深度估计增强的有效性的关键是应该对输出值进行充分的全局分析，因此，2020[127]提出了AdaBins结构，它结合了CNN和transformer。利用transformer出色的全局信息处理能力，结合CNN的局部特征处理能力，深度估计的精度大大提高。

根据[128]，端到端卷积神经网络框架用于车辆测距，以应对由于光变化和视点变化引起的测量误差。该算法基于将RGB信息转换为深度信息，结合检测模块作为输入，并最终基于距离模块预测距离。它的鲁棒性更好，并减少了由于复杂的驾驶环境（如光线不足和遮挡）导致的测距误差。2021，[129]提出了FIERY，一种端到端BEV概率预测模型，该模型将摄像机捕获的当前状态和训练中的未来分布输入到卷积GRU网络进行推理，作为估计深度信息和预测未来多模轨迹的一种方法。

3）传统双目深度估计

与单目相机不同，立体深度估计依赖于平行排列的相机产生的视差。它可以通过找到同一物体的点并进行精确的三角测量来获得场景中可驾驶区域和障碍物的深度信息。尽管没有LIDAR深度估计那么远，但它更便宜，并且可以在有公共视野的情况下重建环境的3D信息。然而，立体摄像机要求摄像机之间的高同步率和采样率，因此技术难点在于立体校准和立体定位，其中，使用最多的是双目摄像头，如图13。

三目相机的工作原理相当于使用两个双目立体视觉系统，如图14所示，它们沿着相同的方向和距离放置。三目立体视觉系统具有窄基线和宽基线。窄基线是左侧和中间相机的线，宽基线是左侧相机和右侧相机的线。窄基线增加了两个相机的共同视野，宽基线在每个可见距离处具有更大的最大视野[144]。三目立体视觉系统的三个摄像头从不同角度拍摄三幅图像，然后使用立体视觉匹配算法获得深度信息！

与单目测距类似，立体测距的工作原理是当实际物体被相机捕捉到图片中时，对其进行仿射变换。该过程包括相机的校准、图像的立体校正、视差图的计算和深度图的计算。由于视差，立体视觉系统需要对不同图像中捕获的对应点进行立体匹配，立体匹配主要分为全局匹配和局部匹配。虽然全局匹配具有较高的精度和较好的鲁棒性，但计算速度较慢，不能满足实时性要求，因此局部匹配主要应用于车辆。

4）基于深度学习的双目深度估计

传统的基于立体的深度估计是通过匹配多幅图像的特征来实现的，尽管进行了广泛的研究，但在处理遮挡、无特征区域或具有重复图案的高纹理区域时，它仍然存在精度差的问题。近年来，基于深度学习的立体深度估计发展迅速，通过使用先验知识来表征特征作为学习任务，深度估计的鲁棒性得到了极大的提高。

2016年，[152]提出MC-CNN通过标记数据来构建训练集，在每个像素点生成正样本和负样本，其中正样本来自具有相同深度的两个图像块，负样本来自具有不同深度的图像块，然后训练神经网络来预测深度。然而，它的计算依赖于局部图像块，这在一些纹理较少或图案重复的区域中引入了较大的误差。因此，2017年[153]提出了GC Net，它对左图像和右图像执行多层卷积和下采样操作，以更好地提取语义特征，然后使用3D卷积处理Cost Vollumn，以提取左图像和右侧图像之间的相关信息以及视差值。2018年，[154]提出了PSMNet，其使用金字塔结构和零卷积来提取多分幅方面信息，并扩展感知领域和多个堆叠的HourGlass结构，以增强3D卷积，从而视差的估计更依赖于不同尺度的信息而不是像素级的局部信息。

因此，可以获得更可靠的视差估计。[155]提出了MVSNet，其利用3D卷积运算成本体积正则化。它首先输出每个深度的概率。然后，它找到深度的加权平均值以获得预测的深度信息，使用多个图像之间的重建约束（光度和几何一致性）来选择预测的正确深度信息。2019年，[156]提出了基于它的P-MVSNet，它通过具有各向同性和各向异性3D卷积的混合3D Unet实现了更好的估计结构。然而，这些网络使用离散点进行深度估计，从而引入误差。[157]考虑到现有立体网络（例如，PSMNet）产生的视差图在几何上不一致，他们提出StereoDRNet，其将几何误差、光度误差和未确定视差作为输入，以产生深度信息并预测遮挡部分。这种方法提供了更好的结果，并显著减少了计算时间。2020年，[158]提出了一种用于连续深度估计的CDN。除了离散点的分布之外，还估计每个点处的偏移，并且离散点和偏移一起形成连续视差估计。

5）基于RGBD测距

RGBD相机通常包含三个相机：彩色相机、红外发射器相机和红外接收器相机，原理如图15所示。与通过视差计算深度的立体相机相比，RGB-D可以主动测量每个像素的深度。此外，基于RGBD传感器的3D重建具有成本效益和准确性，这弥补了单目和立体视觉传感器估计深度信息的计算复杂性和缺乏保证的准确性。

RGB-D测量像素距离，可分为红外结构光法和飞行时间（TOF）法。结构光的原理是红外激光器向物体表面发射一些具有结构特征的图案。然后红外相机将收集由于表面深度不同而产生的图案变化。与依赖于物体本身的特征点的立体视觉不同，结构光方法表征了透射光源，因此，特征点不会随场景而改变，这大大降低了匹配难度。根据不同的编码策略，有时间编码、空间编码和直接编码。时间编码方法可分为二进制码[160]、n值码[161]等。

它具有易于实现、高空间分辨率和高3D测量精度的优点，但测量过程需要投影多个模式，因此它仅适用于静态场景测量。空间编码方法只有一个投影图案，并且图案中每个点的码字是基于其周围相邻点的信息（例如，像素值、颜色或几何）获得的。它适用于动态场景3D信息采集，但在解码阶段丢失空间相邻点信息会导致错误和低分辨率。空间编码分为非正式编码[162]、基于De Bruijn序列的编码[163]和基于M阵列的编码[164]。对每个像素执行直接编码方法。然而，它在相邻像素之间的色差很小，这对噪声非常敏感。它不适用于动态场景，包括[165]提出的灰色直接编码和[166]提出的彩色直接编码。

TOF通过连续向观察到的物体发射光脉冲，然后接收从物体反射回来的光脉冲。根据调制方法的不同，它通常可以分为脉冲调制和连续波调制。测量深度后，RGB-D根据生产时的单个相机位置完成深度和彩色像素之间的配对，并输出一对一的彩色图和深度图。可以在同一图像位置读取颜色信息和距离信息，并且可以计算像素的3D相机坐标以生成点云。然而，RGB-D容易受到日光或其他传感器发出的红外光的干扰，因此不能在室外使用。多个RGB D可以相互干扰，并且在成本和功耗方面具有一些缺点。

视觉SLAM

SLAM（同时定位和映射）分为激光SLAM和视觉SLAM。视觉SLAM使用视觉传感器作为唯一的环境感知传感器。单个视觉传感器的三角测量算法或多个视觉传感器的立体匹配算法可以以良好的精度计算深度信息。同时，由于它包含丰富的颜色和纹理信息，并具有体积小、重量轻、成本低等优点，因此成为当前的研究趋势。视觉SLAM根据视觉传感器类别分为单目视觉SLAM、立体视觉SLAM和RGB-D视觉SLAM。

1）单目SLAM

单目SLAM是一种简单、低成本且易于实现的系统，使用相机作为唯一的外部传感器。根据是否使用概率框架，单眼视觉SLAM分为两种类型。基于概率框架的单眼视觉SLAM构造联合后验概率密度函数，以描述给定从初始时刻到当前时刻的控制输入和观测数据的相机姿态和地图特征的空间位置，由于SLAM应用场景的未知复杂性，其目前被广泛使用。[167]提出了一种基于示例滤波器的SLAM，该SLAM分解运动路径的联合后验分布估计问题，并将其映射为具有示例滤波器的运动路径估计问题和已知路径下的路标估计问题。然而，为了确保定位精度，在复杂的场景和运动中需要更多的粒子，这大大增加了算法的复杂性，重新采样往往会导致样本耗尽和其他问题。[168]通过边缘化每个粒子特征的位置来改进粒子滤波方法，以获得该特征的观测序列用于更新粒子权重的概率，并且不需要状态向量中包括的特征位置。因此，即使在特征密集的环境中，算法的计算复杂度和样本复杂度仍然很低。[169]提出了基于扩展卡尔曼滤波的MonoSLAM，其使用稀疏特征图来表示环境，并通过概率估计来主动搜索和跟踪特征点。

然而，EKFSLAM算法具有高复杂性、较差的数据关联问题和较大的线性化处理误差。[170]提出了FastSLAM，它仍然使用EKF算法来估计环境特征，但通过将移动机器人的姿态表示为粒子并将状态估计分解为采样部分和分辨率部分，计算复杂性大大降低。然而，它使用SLAM的过程模型作为采样粒子的直接重要函数可能会导致粒子退化的问题，这降低了算法的准确性。因此，[171]中提出的FastSLAM2.0使用EKF算法递归地估计移动机器人姿态，获得估计的均值和方差，并使用它们构建高斯分布函数作为重要函数。因此，解决了颗粒降解问题。对于具有非概率框架的单眼视觉SLAM，[172]提出了一种基于关键帧的单眼视力SLAM系统PTAM。

该系统利用一个线程跟踪相机姿态，另一个线程绑定和调整关键帧数据以及所有特征点的空间位置。双线程并行性确保了算法的准确性和计算的效率。[173]提出了基于PTAM的ORB-SLAM，添加了第三个并行线程、环回检测线程和环回检测算法，可以减少SLAM系统产生的累积误差。由于ORB特征的旋转和尺度不变性，保证了每个步骤的内生一致性和良好的鲁棒性。图16显示了两者的比较。[174175]提出了ORB-SLAM2和ORB-SLAM3，并将其扩展到双筒望远镜、RGBD和鱼眼相机。[176]利用相机最近捕获的固定数量的图像作为局部束调整优化的关键帧，以实现SLAM。[177]提出了LIFT-SLAM，它将基于深度学习的特征描述符与传统的基于几何的系统相结合。使用神经网络从图像中提取特征，其基于从学习中获得的特征提供更准确的匹配。

2）双目SLAM

立体视觉SLAM使用多个摄像头作为传感器。由于绝对深度未知，单目SLAM无法获得运动轨迹和地图的真实大小。stereo可以通过视差简单而准确地计算场景中地标的真实3D坐标，然而，它需要更高精度的校准参数，并且成本高昂。在[178]中，通过固定基线提出的LSD-SLAM可以避免单眼SLAM中通常出现的尺度漂移，此外，通过组合两个视差源可以估计欠约束像素的深度。应用于立体视觉SLAM的ORB-SLAM2使用双线程提取左右图像的ORB特征点，然后计算双目视觉特征点并执行匹配。

[179]提出了基于ORB-SLAM 2.0的DMS-SLAM，使用滑动窗口和基于网格的运动统计（GMS）特征匹配方法来查找静态特征位置，并在执行速度上有所提高。然而，基于点特征的算法在低纹理环境中不能很好地工作，因此[180]提出了基于ORB-SLAM2和LSD的PL-SLAM组合点和线特征，这可以保证在更广泛的场景中的鲁棒性能。[181]提出了一种立体视觉双四元数视觉SLAM框架，它使用贝叶斯框架进行姿态估计，对于地图的点云和光流，DQV-SLAM使用ORB函数在动态环境中实现可靠的数据关联。与基于过滤器的方法相比，性能更好。[182]提出了SOFT，一种基于特征跟踪的立体视觉里程计算。SLAM是通过位姿估计和基于特征点的位姿图的构建来实现的，与单目ORB相比，系统的全局一致性得到了保证。然而，当目标距离较远时，双目相机会退化为单目相机，因此，近年来围绕单眼ORB进行了大量的研究。

3）RGB-D SLAM

RGB-D视觉SLAM使用RGB-D传感器作为图像输入设备。该传感器集成了彩色相机和红外相机，以捕获彩色图像和相应的深度图像，因此正逐渐成为SLAM的趋势。[183]从RGB图像中提取特征点，然后将其与深度信息相结合，将特征点逆映射到3D空间，然后使用ICP（点云匹配算法）优化初始姿态。然而，当处于光强度变化的环境中时，RGB-D数据通常缺乏有效性，因此现在将其与IMU传感器计算的状态增量进行融合[184]，以获得更好的结果。当前的深度SLAM由两部分组成，前端和后端。前端从相邻帧之间的图像中估计相机运动，并恢复场景的空间结构，而后端在不同时刻接收视觉里程计输出的相机姿态，以及来自环回检测的信息，并优化它们以获得全局一致的轨迹和地图。[185]使用Kinect RGB-D进行3D环境重建，KinectFusion技术可以将采集的每帧图像数据实时添加到3D地图中。尽管如此，它仍然需要高硬件配置，因为它占用了巨大的内存空间，并且SLAM的性能会长期恶化。[186]基于此提出了RGB-D SLAM问题的改进和优化。系统前端从每个帧的RGB图像中提取特征，结合RANSAC和ICP算法以获得并使用EMM（环境测量模型）模型来验证运动估计，后端基于姿势图优化构建地图。

参考文献

[1] Z. Chen, A. Li, X. Qiu, W. Yuan, and Q. Ge, “Survey of environment visual perception for intelligent vehicle and its supporting key technologies,” J. Hebei Univ. Sci. Technol,vol. 40, 15-23, 2019.

[2] Shengbo Eben Li, Reinforcement Learning for Sequential Decision and Optimal Control. Singapore: Springer Verlag, 2023.

[3] J. Lin, S. Sedigh, and A. Miller, “Modeling Cyber-Physical Systems with Semantic Agents,” in IEEE 34th Annual Computer Software and Applications Conference workshops (COMPSAC-W), 2010: 19 - 23 July 2010, Seoul, Korea ; proceedings, 2010,pp. 13–18.

[4] J. Yao, “Digital image compression arithmetic based on the discrete Fourier transform,”Journal of Shaanxi University of Technology, vol. 28, pp. 22–26, 2012.

[5] E. Elharar, A. Stern, O. Hadar, and B. Javidi, “A Hybrid Compression Method for Integral Images Using Discrete Wavelet Transform and Discrete Cosine Transform,” J.Display Technol., vol. 3, no. 3, pp. 321–325, 2007.

[6] H. Zhang and X. Zhang, “Research on arithmetic of the number theorytransformation(NTT) applied in the image compression,” J. China Coal Soc., vol. 25,pp. 158–164, 2000.

[7] W. Zheng and Y. Piao, “Research on integral (3D) image compression technology based on neural network,” in 5th International Congress on Image and Signal Processing (CISP), 2012: 16-18 Oct. 2012, CQUPT, Chongqing, China, 2012, pp. 382–386.

[8] S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 7, pp. 674–693,1989.

[9] W. M. Lawton, “Tight frames of compactly supported affine wavelets,” Journal of Mathematical Physics, vol. 31, no. 8, pp. 1898–1901, 1990.

[10] L. ChaoFeng et al., “Research on fuzzy image enhancement and restoration effect,” in First International Conference on Electronics Instrumentation & Information Systems (EIIS 2017): June 3-5, Harbin, China, 2017, pp. 1–4.

[11] M. Tico and K. Pulli, “Image enhancement method via blur and noisy image fusion,” in 16th IEEE International Conference on Image Processing (ICIP), 2009: 7-10 Nov. 2009,Cairo, Egypt ; proceedings, 2009, pp. 1521–1524.

[12] L. Chong-Yi, G. Ji-Chang, C. Run-Min, P. Yan-Wei, and W. Bo, “Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior,” IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, vol. 25, no. 12, pp. 5664–5677, 2016.

[13] X. M. Li, “Image enhancement in the fractional Fourier domain,” in 2013 6thInternational Congress on Image and Signal Processing (CISP), 2013, pp. 299–303.

[14] Y. Xian and Y. Tian, “Robust internal exemplar-based image enhancement,” in 2015 IEEE International Conference on Image Processing (ICIP), 2015, pp. 2379–2383.

[15] S. S. Agaian, B. Silver, and K. A. Panetta, “Transform coefficient histogram-based image enhancement algorithms using contrast entropy,” IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, vol. 16, no. 3, pp.741–758, 2007.

[16] S. Song and T. Gao, “Research on image segmentation algorithm based on threshold,” in 2021 13th International Conference on Measuring Technology and Mechatronics Automation: ICMTMA 2021 : proceedings : Beihai, China, 16-17 January 2021, 2021,pp. 306–308.

[17] Y. Zhang and Z. Xia, “Research on the Image Segmentation Based on Improved Threshold Extractions,” in Proceedings of 2018 IEEE 3rd International Conference on Cloud Computing and Internet of things: CCIOT 2018 : October 20-21, 2018, Dalian,China, 2018, pp. 386–389.

[18] N. Zhao, S.-K. Sui, and P. Kuang, “Research on image segmentation method based on weighted threshold algorithm,” in 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Dec.2015 - Dec. 2015, pp. 307–310.

[19] A. Troya-Galvis, P. Gançarski, and L. Berti-Équille, “A Collaborative Framework for Joint Segmentation and Classification of Remote Sensing Images,” in Studies in computational intelligence, 1860-949X, volume 732, Advances in knowledge discovery and management. Volume 7, Cham, Switzerland, 2018, pp. 127–145.

[20] D. Fu, H. Chen, Y. Yang, C. Wang, and Y. Jin, “Image Segmentation Method Based on Hierarchical Region Merging's NCUT,” in 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP): Conference venue: University of Electronic Science and Technology of China, Sichuan Province, China, conference dates: Dec. 14-16 2018, 2018, pp. 63–68.

[21] G. Huang and C.-M. Pun, “Interactive Segmentation Based on Initial Segmentation and Region Merging,” in 2013 10th International Conference Computer Graphics, Imaging and Visualization, 2013, pp. 52–55.

[22] X. Li and L. Lu, “An Improved Region Growing Method for Segmentation,” in 2012 International Conference on Computer Science and Service System: CSSS 2012 : 11-13 August 2012 : Nanjing, China : proceedings, 2012, pp. 2313–2316.

[23] P. Sun and L. Deng, “An image fusion method based on region segmentation and wavelet transform,” in 20th International Conference on Geoinformatics (GEOINFORMATICS), 2012, Jun. 2012 - Jun. 2012, pp. 1–5.

[24] S. Qiang, L. Guoying, M. Jingqi, and Z. Hongmei, “An edge-detection method based on adaptive canny algorithm and iterative segmentation threshold,” in ICCSSE:Proceedings of 2016 2nd International Conference on Control Science and SystemsEngineering : July 27-29, 2016, Singapore, 2016, pp. 64–67.

[25] K. Strzecha, A. Fabijanska, and D. Sankowski, “Application Of The Edge-Based Image Segmentation,” in Proceedings of the 2nd International Conference on PerspectiveTechnologies and Methods in MEMS Design, 2006, pp. 28–31.

[26] L. Xiang, W. Xiaoqing, and Y. Xiong, “Remaining edges linking method of motion segmentation based on edge detection,” in 9th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), 2012, May. 2012 - May. 2012, pp. 1895–1899.

[27] E. Saber, A. M. Tekalp, and G. Bozdagi, “Fusion of color and edge information for improved segmentation and edge linking,” in The 1996 IEEE international conference on acoustics, speech, and signal processing, 1996, pp. 2176–2179.

[28] G. Hao, L. Min, and H. Feng, “Improved Self-Adaptive Edge Detection Method Based on Canny,” in International conference on intelligent human-machine systems and cybernetics, 2013, pp. 527–530.

[29] A. Rosenfeld, “The Max Roberts Operator is a Hueckel-Type Edge Detector,” IEEE transactions on pattern analysis and machine intelligence, vol. 3, no. 1, pp. 101–103,1981.

[30] N. R. Pal and S. K. Pal, “A review on image segmentation techniques,” Pattern Recognition, vol. 26, no. 9, pp. 1277–1294, 1993.

[31] T. Sanida, A. Sideris, and M. Dasygenis, “A Heterogeneous Implementation of the Sobel Edge Detection Filter Using OpenCL,” in 2020 9th International Conference on Modern Circuits and Systems Technologies (MOCAST), 2020, pp. 1–4.

[32] L. Pei, Z. Xie, and J. Dai, “Joint edge detector based on Laplacian pyramid,” in 3rd International Congress on Image and Signal Processing (CISP), 2010: 16-18 Oct. 2010,Yantai, China ; proceedings, 2010, pp. 978–982.

[33] Satinder Chopra and Vladimir Alexeev, “Application of texture attribute analysis to 3D seismic data,” SEG Technical Program Expanded Abstracts 2005, vol. 25, no. 8, pp.767–770, 2005.

[34] J. Li, “Application of Various Cameras in L3 Level Autonomous Driving Visual Perception,” Auto Electric Parts, vol. 8, pp. 11–12, 2019.

[35] J. Wang and H. Liu, “Vehicle navigation method based on trinocular vision,” Journal of Computer Applications, vol. 34, pp. 1762–1773, 2014.

[36] D. Seo, H. Park, K. Jo, K. Eom, S. Yang, and T. Kim, “Omnidirectional stereo vision based vehicle detection and distance measurement for driver assistance system,” in IECON 2013: 39th annual conference of the IEEE Industrial Electronics Societ ; 10-13 Nov. 2013, Vienna, Austria ; proceedings, 2013, pp. 5507–5511.

[37] J. Niu, J. Lu, M. Xu, P. Lv, and X. Zhao, “Robust Lane Detection using Two-stage Feature Extraction with Curve Fitting,” Pattern Recognition, vol. 59, pp. 225–233,2016.

[38] M. S. Sravan, S. Natarajan, E. S. Krishna, and B. J. Kailath, “Fast and accurate on-road vehicle detection based on color intensity segregation,” Procedia Computer Science,vol. 133, pp. 594–603, 2018.

[39] M. Anandhalli and V. P. Baligar, “A novel approach in real-time vehicle detection and tracking using Raspberry Pi,” Alexandria Engineering Journal, vol. 57, no. 3, pp. 1597–1607, 2018.

[40] A. R. Webb, Statistical pattern recognition. London: Arnold, 1999.

[41] B. W. White and F. Rosenblatt, “Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms,” The American Journal of Psychology, vol. 76, no. 4, p.705, 1963.

[42] L. Chen and K.-H. Yap, “A fuzzy K-nearest-neighbor algorithm to blind image deconvolution,” in System security and assurance, 2003, pp. 2049–2054.

[43] B. D. Maciel and R. A. Peters, “A comparison of neural and statistical techniques in object recognition,” in 2000 IEEE international conference on systems, man and cybernetics, 2000, pp. 2833–2838.

[44] K. Kato, M. Suzuki, Y. Fujita, and F. Y, “Image synthesis display method and apparatus for vehicle camera,” U.S. Patent 7 139 412 B2, Nov 21, 2006.

[45] J. Zhang, Z. Liu, Z. Qi, and Y. Ma, “Development of the Adaptive Cruise Control for Cars,” Vehicle & Power Technology, vol. 2, pp. 45–49, 2003.

[46] C.-L. Su, C.-J. Lee, M.-S. Li, and K.-P. Chen, “3D AVM system for automotive applications,” in 2015 10th International Conference on Information, Communications and Signal Processing (ICICS): 2-4 Dec. 2015, 2015, pp. 1–5.

[47] Z. Zhang, “Flexible camera calibration by viewing a plane from unknown orientations,”in The proceedings of the seventh IEEE international conference on computer vision,1999, 666-673 vol.1.

[48] Y. I. Abdel-Aziz and H. M. Karara, “Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates in Close-Range Photogrammetry,”photogramm eng remote sensing, vol. 81, no. 2, pp. 103–107, 2015.

[49] R. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses,” IEEE J. Robot. Automat., vol. 3,no. 4, pp. 323–344, 1987.

[50] D. Scaramuzza, A. Martinelli, and R. Siegwart, “A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion,” in IEEE International Conference on Computer Vision Systems, 2006: ICVS '06 ; 04 - 07 Jan. 2006, [NewYork, New York], 2006, p. 45.

[51] X. Ren and Z. Lin, “Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures,” Int J Comput Vis, vol. 104, no. 1, pp. 1–14, 2013.

[52] D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int J Comput Vis, vol. 60, no. 2, pp. 91–110, 2004.

[53] L. Zhu, Y. Wang, B. Zhao, and X. Zhang, “A Fast Image Stitching Algorithm Based on Improved SURF,” in 2014 Tenth International Conference on Computational Intelligence and Security, 2014, pp. 171–175.

[54] Q. Liu, “Research on the Around View Image System of Engineering Vehicle,” China University of Mining and Technology, Xuzhou, Jiangsu, 2017.

[55] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017.

[56] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science (New York, N.Y.), vol. 313, no. 5786, pp. 504–507, 2006.

[57] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on: Date, 7-12 June 2015, 2015, pp. 3431–3440.

[58] M. D. Zeiler and R. Fergus, “Visualizing and Understanding Convolutional Networks,”in 2014, pp. 818–833.

[59] C. Szegedy et al., “Going deeper with convolutions,” in Computer Vision and PatternRecognition (CVPR), 2015 IEEE Conference on: Date, 7-12 June 2015, 2015, pp. 1–9.

[60] M. Lin, Q. Chen, and S. Yan, “Network In Network,” 2013. [Online]. Available: https://arxiv.org/pdf/1312.4400

[61] S. S. Saini and P. Rawat, “Deep Residual Network for Image Recognition,” in 2022 IEEE International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), Apr. 2022 - Apr. 2022, pp. 1–4.

[62] H. Noh, S. Hong, and B. Han, “Learning Deconvolution Network for Semantic Segmentation,” in 2015 IEEE International Conference on Computer Vision: 11-18 December 2015, Santiago, Chile : proceedings, 2015, pp. 1520–1528.

[63] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab:Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2018.

[64] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017, pp. 2261–2269.

[65] S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, “Aggregated Residual Transformations for Deep Neural Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017, pp. 5987–5995.

[66] L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking Atrous Convolution for Semantic Image Segmentation,” 2017. [Online]. Available: https://arxiv.org/pdf/1706.05587

[67] A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” 2017. [Online]. Available: https://arxiv.org/pdf/1704.04861

[68] X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018 - Jun. 2018, pp. 6848–6856.

[69] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design,” in LNCS sublibrary: SL6 - Image processing,computer vision, pattern recognition, and graphics, 11205-11220, Computer Vision -ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018 :proceedings / Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss (eds.),Cham, Switzerland, 2018, pp. 122–138.

[70] K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, “GhostNet: More Features FromCheap Operations,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2020 - Jun. 2020, pp. 1577–1586.

[71] T. Takikawa, D. Acuna, V. Jampani, and S. Fidler, “Gated-SCNN: Gated Shape CNNs for Semantic Segmentation,” in 2019, pp. 5229–5238.

[72] K. He, X. Zhang, S. Ren, and J. Sun, “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 9, pp. 1904–1916, 2015.

[73] R. Girshick, “Fast R-CNN,” in 2015 IEEE International Conference on Computer Vision: 11-18 December 2015, Santiago, Chile : proceedings, 2015, pp. 1440–1448.

[74] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1137–1149, 2017.

[75] J. Dai, Y. Li, K. He, and J. Sun, “R-FCN: Object Detection via Region-based Fully Convolutional Networks,” Advances in Neural Information Processing Systems, vol. 29, 2016.

[76] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified,Real-Time Object Detection,” in 2016, pp. 779–788.

[77] J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017,pp. 6517–6525.

[78] J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” 2018. [Online].Available: https://arxiv.org/pdf/1804.02767

[79] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” Apr. 2020. [Online]. Available: http://arxiv.org/pdf/2004.10934v1

[80] X. Zhu, S. Lyu, X. Wang, and Q. Zhao, “TPH-YOLOv5: Improved YOLOv5 Based onTransformer Prediction Head for Object Detection on Drone-captured Scenarios,” 2021.[Online]. Available: https://arxiv.org/pdf/2108.11539

[81] W. Liu et al., “SSD: Single Shot MultiBox Detector,” in 2016, pp. 21–37.

[82] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal Loss for Dense Object Detection,” in 2017, pp. 2980–2988.

[83] M. Tan, R. Pang, and Q. V. Le, “EfficientDet: Scalable and Efficient Object Detection,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),Jun. 2020 - Jun. 2020, pp. 10778–10787.

[84] N. Shazeer, K. Fatahalian, W. R. Mark, and R. T. Mullapudi, “HydraNets: SpecializedDynamic Architectures for Efficient Inference,” in CVPR 2018: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings : 18-22 June2018, Salt Lake City, Utah, 2018, pp. 8080–8089.

[85] R. Wang, B. Li, J. Chu, and S. Ji, “Study on the Method of Measuring the Leading Vehicle Distance Based on the On-board Monocular Camera,” J. Highw. Transp. Res. Dev. (Chin. Ed.), vol. 18, pp. 94–98, 2001.

[86] C.-F. Wu, C.-J. Lin, and C.-Y. Lee, “Applying a Functional Neurofuzzy Network to Real-Time Lane Detection and Front-Vehicle Distance Measurement,” IEEE Trans. Syst., Man, Cybern. C, vol. 42, no. 4, pp. 577–589, 2012.

[87] R. Adamshuk et al., “On the applicability of inverse perspective mapping for the forward distance estimation based on the HSV colormap,” in Proceedings: Hilton Toronto, Toronto, Canada, 23-25 March, 2017, 2017, pp. 1036–1041.

[88] S. Tuohy, D. O'Cualain, E. Jones, and M. Glavin, “Distance determination for an automobile environment using inverse perspective mapping in OpenCV,” in IET Irish Signals and Systems Conference (ISSC 2010), Jun. 2010, pp. 100–105.

[89] W. Yang, L. Wei, J. Gong, and Q. Zhang, “Research on etection of Longitudinal VehicleSpacing Based on Monocular Vision,” Computer Measurement and Control, vol. 20, pp.2039–2044, 2012.

[90] C. Guan, L. Wei, J. Qiao, and W. Yang, “A vehicle distance measurement method with monocular vision based on vanishing point,” Electronic Measurement Technology, vol.41, pp. 83–87, 2018.

[91] J. Liu, S. Hou, K. Zhang, and X. Yan, “Vehicle distance measurement with implementation of vehicle attitude angle estimation and inverse perspective mapping based on monocular vision,” Trans. Chin. Soc. Agric. Eng., vol. 34, pp. 70–76, 2018.

[92] B. Li, X. Zhang, and M. Sato, “Pitch angle estimation using a Vehicle-Mounted monocular camera for range measurement,” in ICSP2014: 2014 IEEE 12th International Conference on Signal Processing proceedings : October 19-23, 2014, HangZhou, China,2014, pp. 1161–1168.

[93] N. Snavely, S. M. Seitz, and R. Szeliski, “Photo tourism: exploring photo collections in 3D,” ACM Trans. Graph., vol. 25, no. 3, pp. 835–846, 2006.

[94] H. Cui, S. Shen, X. Gao, and Z. Hu, “Batched Incremental Structure-from-Motion,” in2017 International Conference on 3D Vision: 3DV 2017 : Qingdao, Canada, 10-12 October 2017 : proceedings, 2017, pp. 205–214.

[95] N. Snavely, S. M. Seitz, and R. Szeliski, “Modeling the World from Internet Photo Collections,” Int J Comput Vis, vol. 80, no. 2, pp. 189–210, 2008.

[96] C. Wu, “Towards Linear-Time Incremental Structure from Motion,” in 3DV 2013: 2013International Conference on 3D Vision : proceedings : 29 June-1 July 2013, Seattle,Washington, USA, 2013, pp. 127–134.

[97] P. Moulon, P. Monasse, and R. Marlet, “Adaptive Structure from Motion with a Contrario Model Estimation,” in LNCS sublibrary: SL 6 - Image processing, computer vision, pattern recognition, and graphics, 7724-7727, Computer vision - ACCV 2012:11th Asian Conference on Computer Vision, Daejeon, Korea, November 5-9, 2012 :revised selected papers / Kyoung Mu Lee, Yasuyuki Matsushita, James M. Rehg, ZhanyiHu (eds.), Heidelberg, 2013, pp. 257–270.

[98] H. Cui, S. Shen, W. Gao, and Z. Wang, “Progressive Large-Scale Structure-from-Motionwith Orthogonal MSTs,” in 2018 International Conference on 3D Vision: 3DV 2018 :Verona, Italy, 5-8 September 2018 : proceedings, 2018, pp. 79–88.

[99] N. Jiang, Z. Cui, and P. Tan, “A Global Linear Method for Camera Pose Registration,”in 2013 IEEE International Conference on Computer Vision: 1-8 December 2013, 2014,pp. 481–488.

[100] Z. Cui and P. Tan, “Global Structure-from-Motion by Similarity Averaging,” in 2015 IEEE International Conference on Computer Vision: 11-18 December 2015, Santiago,Chile : proceedings, 2015, pp. 864–872.

[101] O. Ozyesil and A. Singer, “Robust camera location estimation by convex programming,” in Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on: Date, 7-12 June 2015, 2015, pp. 2674–2683.

[102] C. Sweeney, T. Sattler, T. Hollerer, M. Turk, and M. Pollefeys, “Optimizing the Viewing Graph for Structure-from-Motion,” in 2015 IEEE International Conference on Computer Vision: 11-18 December 2015, Santiago, Chile : proceedings, 2015, pp. 801–809.4577.

[103] S. Zhu et al., “Very Large-Scale Global SfM by Distributed Motion Averaging,” in CVPR 2018: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings : 18-22 June 2018, Salt Lake City, Utah, 2018, pp. 4568–

[104] B. Bhowmick, S. Patra, A. Chatterjee, V. M. Govindu, and S. Banerjee, “Divide and Conquer: Efficient Large-Scale Structure from Motion Using Graph Partitioning,” in Image Processing, Computer Vision, Pattern Recognition, and Graphics, vol. 9004, Computer Vision -- ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Revised Selected Papers, Part II, Cham, 2015, pp.273–287.

[105] R. Gherardi, M. Farenzena, and A. Fusiello, “Improving the efficiency of hierarchical structure-and-motion,” in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on: Date, 13-18 June 2010, 2010, pp. 1594–1600.33

[106] M. Farenzena, A. Fusiello, and R. Gherardi, “Structure-and-motion pipeline on a hierarchical cluster tree,” in 2009 IEEE 12th International Conference on Computer Vision Workshops, 2009, pp. 1489–1496.

[107] R. Toldo, R. Gherardi, M. Farenzena, and A. Fusiello, “Hierarchical structure-and-motion recovery from uncalibrated images,” Computer Vision and ImageUnderstanding, vol. 140, pp. 127–143, 2015.

[108] H. Cui, X. Gao, S. Shen, and Z. Hu, “HSfM: Hybrid Structure-from-Motion,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017, pp. 2393–2402.

[109] S. Zhu et al., “Parallel Structure from Motion from Local Increment to Global Averaging,” Feb. 2017. [Online]. Available: http://arxiv.org/pdf/1702.08601v3

[110] S. Vijayanarasimhan, S. Ricco, C. Schmid, R. Sukthankar, and K. Fragkiadaki, “SfM-Net: Learning of Structure and Motion from Video,” 2017. [Online]. Available: https://arxiv.org/pdf/1704.07804

[111] T. Zhou, M. Brown, N. Snavely, and D. G. Lowe, “Unsupervised Learning of Depthand Ego-Motion from Video,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017, pp. 6612–6619.

[112] J. L. Schonberger and J.-M. Frahm, “Structure-from-Motion Revisited,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4104–4113.

[113] Z. Feng, “FCWs Test and Evaluation Based on Monicular Camera,” Special Purpose Vehicle, vol. 9, pp. 50–53, 2021.

[114] S. Bougharriou, F. Hamdaoui, and A. Mtibaa, “Vehicles distance estimation using detection of vanishing point,” EC, vol. 36, no. 9, pp. 3070–3093, 2019.

[115] F. Ma, J. Shi, L. Ge, K. Dai, S. Zhong, and L. Wu, “Progress in research on monocular visual odometry of autonomous vehicles,” J. Jilin Univ., Eng. Technol. Ed.,vol. 50, pp. 765–775, 2020.

[116] A. Saxena, S. Chung, and A. Ng, “Learning Depth from Single Monocular Images,”Advances in Neural Information Processing Systems, vol. 18, 2005.

[117] D. Eigen, C. Puhrsch, and R. Fergus, “Depth Map Prediction from a Single Image using a Multi-Scale Deep Network,” Jun. 2014. [Online]. Available: http://arxiv.org/pdf/1406.2283v1

[118] D. Eigen and R. Fergus, “Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture,” in 2015 IEEE International Conference on Computer Vision: 11-18 December 2015, Santiago, Chile : proceedings,2015, pp. 2650–2658.

[119] D. Bao and P. Wang, “Vehicle distance detection based on monocular vision,” in Proceedings of the 2016 IEEE International Conference on Progress in Informatics and Computing: PIC 2016 : December 23-25, 2016, Shanghai, China, 2016, pp. 187–191.

[120] H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, “Deep Ordinal Regression Network for Monocular Depth Estimation,” Proceedings. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2018, pp. 2002–2011,2018.

[121] Y. Wang, W.-L. Chao, D. Garg, B. Hariharan, M. Campbell, and K. Q. Weinberger,“Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D ObjectDetection for Autonomous Driving,” Dec. 2018. [Online]. Available:http://arxiv.org/pdf/1812.07179v6

[122] Zengyi Qin, Jinglu Wang, and Yan Lu, “MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization,” AAAI, vol. 33, no. 01, pp. 8851–8858,2019.

[123] I. Barabanau, A. Artemov, E. Burnaev, and V. Murashkin, “Monocular 3D ObjectDetection via Geometric Reasoning on Keypoints,” 2019. [Online]. Available: https://arxiv.org/pdf/1905.05618

[124] Y. Kim and D. Kum, “Deep Learning based Vehicle Position and Orientation Estimation via Inverse Perspective Mapping Image,” in 2019 IEEE Intelligent Vehicles Symposium (IV 2019): Paris, France 9-12 June 2019, 2019, pp. 317–323.

[125] d. xu, E. Ricci, W. Ouyang, X. Wang, and N. Sebe, “Monocular Depth Estimation Using Multi-Scale Continuous CRFs as Sequential Deep Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 6, pp. 1426–1440, 2019.

[126] A. Kundu, Y. Li, and J. M. Rehg, “3D-RCNN: Instance-Level 3D ObjectReconstruction via Render-and-Compare,” in CVPR 2018: 2018 IEEE/CVF Conferenceon Computer Vision and Pattern Recognition : proceedings : 18-22 June 2018, Salt Lake City, Utah, 2018, pp. 3559–3568.

[127] S. F. Bhat, I. Alhashim, and P. Wonka, “AdaBins: Depth Estimation Using AdaptiveBins,” in 2021, pp. 4009–4018.

[128] C. Shen, X. Zhao, Z. Liu, T. Gao, and J. Xu, “Joint vehicle detection and distance prediction via monocular depth estimation,” IET Intelligent Transport Systems, vol. 14,no. 7, pp. 753–763, 2020.

[129] A. Hu et al., “FIERY: Future Instance Prediction in Bird's-Eye View From SurroundMonocular Cameras,” in 2021, pp. 15273–15282.

[130] Y. Cao, Z. Wu, and C. Shen, “Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 11, pp. 3174–3182, 2018.

[131] B. Li, Y. Dai, H. Chen, and M. He, “Single image depth estimation by dilated deepresidual convolutional neural network and soft-weight-sum inference,” 2017. [Online]. Available: https://arxiv.org/pdf/1705.00534

[132] I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, “Deeper Depth Prediction with Fully Convolutional Residual Networks,” in 2016 Fourth International Conference on 3D Vision (3DV), 2016, pp. 239–248.

[133] J.-H. Lee, M. Heo, K.-R. Kim, and C.-S. Kim, “Single-Image Depth Estimation Based on Fourier Domain Analysis,” in CVPR 2018: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings : 18-22 June 2018, Salt Lake City, Utah, 2018, pp. 330–339.

[134] A. Chakrabarti, J. Shao, and G. Shakhnarovich, “Depth from a Single Image by Harmonizing Overcomplete Local Network Predictions,” Advances in Neural Information Processing Systems, vol. 29, 2016.

[135] F. Liu, C. Shen, and G. Lin, “Deep convolutional neural fields for depth estimation from a single image,” in Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on: Date, 7-12 June 2015, 2015, pp. 5162–5170.

[136] D. Xu, E. Ricci, W. Ouyang, X. Wang, and N. Sebe, “Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017 - Jul. 2017, pp. 161–169.

[137] D. Xu, W. Wang, H. Tang, H. Liu, N. Sebe, and E. Ricci, “Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation,” in CVPR 2018: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition :proceedings : 18-22 June 2018, Salt Lake City, Utah, 2018, pp. 3917–3925.

[138] C. Godard, O. M. Aodha, and G. J. Brostow, “Unsupervised Monocular Depth Estimation with Left-Right Consistency,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6602–6611.

[139] J. Watson, M. Firman, G. Brostow, and D. Turmukhambetov, “Self-Supervised Monocular Depth Hints,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 2162–2171.

[140] Y. Luo et al., “Single View Stereo Matching,” in CVPR 2018: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings : 18-22 June 2018, Salt Lake City, Utah, 2018, pp. 155–163.

[141] R. Garg, V. K. B.G., G. Carneiro, and I. Reid, “Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue,” in 2016, pp. 740–756.

[142] W. Kay et al., “The Kinetics Human Action Video Dataset,” 2017. [Online]. Available: https://arxiv.org/pdf/1705.06950

[143] V. Casser, S. Pirk, R. Mahjourian, and A. Angelova, “Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos,”2018. [Online]. Available: https://arxiv.org/pdf/1811.06152

[144] Y. Tan, S. Goddard, and L. C. Pérez, “A prototype architecture for cyber-physical systems,” SIGBED Rev., vol. 5, no. 1, pp. 1–2, 2008.

[145] A. L. Hou, J. Chen, Z. J. Jin, Q. Liao, and Y. Geng, “Binocular Vision Measurement of Distance Based on Vehicle Logo Location,” AMM, 229-231, pp. 1154–1157, 2012.

[146] ZHANG Z B, LIU S S, XU G, et al., “A Vehicle Distance Measurement Based on Binocular Stereo Vision,” JATIT, vol. 44, pp. 179–184, 2012.

[147] J. j. LI, “Analysis of Pixel-level Remote Sensing Image Fusion Methods,” Geo-information Science, vol. 10, no. 1, pp. 128–134, 2008.

[148] N. Shao, H.-G. Li, L. Liu, and Z.-L. Zhang, “Stereo Vision Robot Obstacle Detection Based on the SIFT,” in 2010 Second WRI Global Congress on Intelligent Systems, Dec.2010 - Dec. 2010, pp. 274–277.

[149] Y. Pan and C. Wu, “Research on Integrated Location with Stereo Vision and Map for Intelligent Vehicle,” Journal of Hubei University of Technology, vol. 32, pp. 55–59,2017.

[150] X. Huang, F. Shu, and W. Cao, “Research on Distance Measurement Method of Front Vehicle Based on Binocular Vision,” Automobile Technology, vol. 12, pp. 16–21, 2016.

[151] Y. Wei et al., “SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation,” 2022. [Online]. Available: https://arxiv.org/pdf/2204.03636

[152] Jure Žbontar and Yann LeCun, “Stereo matching by training a convolutional neural network to compare image patches,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 2287–2318, 2016.

[153] A. Kendall et al., “End-to-End Learning of Geometry and Context for Deep StereoRegression,” in 2017 IEEE International Conference on Computer Vision: ICCV 2017 :proceedings : 22-29 October 2017, Venice, Italy, 2017, pp. 66–75.

[154] J. -R. Chang and Y. -S. Chen, “Pyramid Stereo Matching Network,” in 2018IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 5410–5418.

[155] Y. Yao, Z. Luo, S. Li, T. Fang, and L. Quan, “MVSNet: Depth Inference for Unstructured Multi-view Stereo,” in LNCS sublibrary: SL6 - Image processing,computer vision, pattern recognition, and graphics, 11205-11220, Computer Vision -ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018 :proceedings / Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss (eds.),Cham, Switzerland, 2018, pp. 785–801.

[156] K. Luo, T. Guan, L. Ju, H. Huang, and Y. Luo, “P-MVSNet: Learning Patch-WiseMatching Confidence Aggregation for Multi-View Stereo,” in 2019 IEEE/CVFInternational Conference on Computer Vision (ICCV), 2019, pp. 10451–10460.

[157] R. Chabra, J. Straub, C. Sweeney, R. Newcombe, and H. Fuchs, “StereoDRNet:Dilated Residual StereoNet,” in Proceedings of the IEEE/CVF Conference, 2019, pp.11786–11795.

[158] D. Garg, Y. Wang, B. Hariharan, M. Campbell, K. Q. Weinberger, and W.-L. Chao,“Wasserstein Distances for Stereo Disparity Estimation,” Advances in NeuralInformation Processing Systems, vol. 33, pp. 22517–22529, 2020.

[159] Y. Chen, W. Zuo, K. Wang, and Q. Wu, “Survey on Structured Light PatternCodification Methods,” Journal of Chinese Computer Systems, vol. 31, pp. 1856–1863,2010.

[160] J.L. Posdamer and M.D. Altschuler, “Surface measurement by space-encodedprojected beam systems,” Computer Graphics and Image Processing, vol. 18, no. 1, pp.1–17, 1982.

[161] E. Horn and N. Kiryati, “Toward optimal structured light patterns,” in Internationalconference on recent advances in 3-D digital imaging and modeling, 1997, pp. 28–35.

[162] T. P. Koninckx and L. van Gool, “Real-time range acquisition by adaptive structuredlight,” IEEE transactions on pattern analysis and machine intelligence, vol. 28, no. 3,pp. 432–445, 2006.

[163] J. Pages, J. Salvi, and J. Forest, “A new optimised De Bruijn coding strategy forstructured light patterns,” in Proceedings of the 17th International Conference onPattern Recognition: ICPR 2004, 2004, 284-287 Vol.4.

[164] P. M. Griffin and S. R. Yee, “The use of a uniquely encoded light pattern for rangedata acquisition,” Computers & Industrial Engineering, vol. 21, 1-4, pp. 359–363, 1991.

[165] A. K.C. Wong, P. Niu, and X. He, “Fast acquisition of dense depth data by a new structured light scheme,” Computer Vision and Image Understanding, vol. 98, no. 3, pp. 398–422, 2005.

[166] J. Tajima and M. Iwakawa, “3-D data acquisition by Rainbow Range Finder,” in Pattern recognition: 10th International conference : Papers, 1990, pp. 309–313.

[167] G. Grisetti, C. Stachniss, and W. Burgard, “Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters,” IEEE Trans. Robot., vol. 23, no. 1, pp. 34–46,2007.

[168] T. Yap, M. Li, A. I. Mourikis, and C. R. Shelton, “A particle filter for monocular vision-aided odometry,” in 2011 IEEE International Conference on Robotics and Automation: (ICRA) ; 9-13 May 2011, Shanghai, China, 2011, pp. 5663–5669.

[169] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “MonoSLAM: real-time single camera SLAM,” IEEE transactions on pattern analysis and machine intelligence, vol. 29, no. 6, pp. 1052–1067, 2007.

[170] R. Sim, P. Elinas, and J. J. Little, “A Study of the Rao-Blackwellised Particle Filter for Efficient and Accurate Vision-Based SLAM,” Int J Comput Vis, vol. 74, no. 3, pp. 303–318, 2007.

[171] M. Montemerlo, S. Thrun, D. Koller, and B. Wegbreit, “FastSLAM 2.0: an improved particle filtering algorithm for simultaneous localization and mapping that provably converges,” proc.int.conf.on artificial intelligence, 2003.

[172] G. Klein and D. Murray, “Parallel Tracking and Mapping for Small AR Workspaces,” in 2007 6th ieee & acm international symposium on mixed and augmented reality, 2008, pp. 1–10. [173] R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM: A Versatile and Accurate Monocular SLAM System,” IEEE Trans. Robot., vol. 31, no. 5, pp. 1147– 1163, 2015.

[174] R. Mur-Artal and J. D. Tardos, “ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras,” IEEE Trans. Robot., vol. 33, no. 5, pp. 1255– 1262, 2017.

[175] C. Campos, R. Elvira, J. J. G. Rodriguez, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM,” IEEE Trans. Robot., vol. 37, no. 6, pp. 1874–1890, 2021.

[176] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, “Monocular Vision Based SLAM for Mobile Robots,” in ICPR 2006: The 18th International Conference on Pattern Recognition : 20-24 August 2006, Hong Kong : [conference proceedings], 2006,pp. 1027–1031.

[177] H. M. S. Bruno and E. L. Colombini, “LIFT-SLAM: A deep-learning feature-based monocular visual SLAM method,” Neurocomputing, vol. 455, pp. 97–110, 2021.

[178] J. Engel, J. Stuckler, and D. Cremers, “Large-scale direct SLAM with stereo cameras,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): Sept. 28, 2015 - Oct. 2, 2015, Hamburg, Germany, 2015, pp. 1935– 1942.

[179] G. Liu, W. Zeng, B. Feng, and F. Xu, “DMS-SLAM: A general visual SLAM system for dynamic scenes with multiple sensors,” Sensors, vol. 19, no. 17, p. 3714, 2019.

[180] R. Gomez-Ojeda, F. -A. Moreno, D. Zuñiga-Noël, D. Scaramuzza, and J. Gonzalez-Jimenez, “PL-SLAM: A Stereo SLAM System Through the Combination of Points andLine Segments,” IEEE Transactions on Robotics, vol. 35, no. 3, pp. 734–746, 2019.

[181] S. Bultmann, K. Li, and U. D. Hanebeck, “Stereo Visual SLAM Based on Unscented Dual Quaternion Filtering,” in 2019 22th International Conference on Information Fusion (FUSION), 2019, pp. 1–8.

[182] I. Cvišić, J. Ćesić, I. Marković, and I. Petrović, “SOFT-SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles,” J. Field Robotics, vol. 35, no. 4, pp. 578–595, 2018.

[183] P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox, “RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments,” The International Journal of Robotics Research, vol. 31, no. 5, pp. 647–663, 2012.

[184] L. Furler, V. Nagrath, A. S. Malik, and F. Meriaudeau, “An Auto-Operated Telepresence System for the Nao Humanoid Robot,” in 2013 International Conference on Communication Systems and Network Technologies: Proceedings, 6-8 April, 2013,Gwalior, India ; [edited by Geetam S. Tomar, Manish Dixit, Frank Z. Wang], 2013, pp.262–267.

[185] R. A. Newcombe et al., “KinectFusion: Real-time dense surface mapping and tracking,” in 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2011: 26 - 29 Oct. 2011, Basel, Switzerland, 2011, pp. 127–136.

[186] F. Endres, J. Hess, J. Sturm, D. Cremers, and W. Burgard, “3-D Mapping With an RGB-D Camera,” IEEE Trans. Robot., vol. 30, no. 1, pp. 177–187, 2014.

[187] M. Yang, “Influence of Weather Environment on Drivers' Hazard Perception Characteristics Based on Simulated Driving,” Beijing Automotive Engineering, vol. 2,pp. 11–13, 2017.

[188] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer,“SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size,” 2016. [Online]. Available: https://arxiv.org/pdf/1602.07360.

下滑查看更多参考文献

转载自自动驾驶之心，文中观点仅供分享交流，不代表本公众号立场，如涉及版权等问题，请您告知，我们将及时处理。

-- END --

登录阅读全文



免责声明：该内容由专栏作者授权发布或作者转载，目的在于传递更多信息，并不代表本网赞同其观点，本站亦不保证或承诺内容真实性等。若内容或图片侵犯您的权益，请及时联系本站删除。侵权投诉联系： nick.zong@aspencore.com！

智驾最前沿「智驾最前沿」深耕自动驾驶领域技术、资讯等信息，解读行业现状、紧盯行业发展、挖掘行业前沿，致力于助力自动驾驶发展与落地！公众号：智驾最前沿

进入专栏

文章：1948篇粉丝：36人

 私信