Methods, systems, and devices for providing a virtual ramp preview to a driver of a vehicle

The present disclosure relates to methods, systems, and/or devices for providing a virtual ramp preview to a driver of a vehicle.

Various vehicles (e.g., vans or minivans) may be equipped with a deployable ramp that deploys from a side or a back of a vehicle to assist passengers (e.g., a person using a wheelchair) with entering or exiting the vehicle. Currently, a driver of such a vehicle must check manually to ensure there is sufficient space for deploying the ramp when looking for a place to park the vehicle. Parking spaces or locations may have obstacles and/or objects (e.g., a curb, a pile of snow, a pole, a puddle, an uneven surface, etc.) that interferes with the ability of the ramp to extend outward from the vehicle and with a passenger's ability to ascend or descend the ramp safely.

Accordingly, it is desirable to provide methods, systems, and devices for providing a virtual ramp preview to a driver of a vehicle.

In general, one aspect of the subject matter described in this disclosure may be embodied in a virtual ramp preview system for a vehicle. The virtual ramp preview system may include a deployable ramp. The deployable ramp may deploy from the vehicle. The virtual ramp preview system may further include one or more cameras coupled to an exterior of the vehicle. The one or more cameras may provide one or more fields of view of a surrounding area of the vehicle. The virtual ramp preview system may further include a user interface located within the vehicle and having a display. The virtual ramp preview system may further include an electronic control unit (ECU) electrically connected to the one or more cameras and the user interface. The ECU may display, via the user interface, a composite image of the one or more fields of view. The composite image may include a virtual vehicle that is a virtual representation of the vehicle. The ECU may augment the composite image to include a virtual ramp extending from the virtual vehicle to assist a driver of the vehicle with determining an amount of space that is required to deploy the deployable ramp.

In one aspect, the subject matter may be embodied in a virtual ramp preview system for a vehicle having a deployable wheelchair ramp. The virtual ramp preview system may include one or more cameras located on an exterior of the vehicle. The one or more cameras may provide real-time video data including one or more fields of view of a surrounding area of the vehicle. The virtual ramp preview system may further include a user interface located within the vehicle and having a display. The virtual ramp preview system may further include an electronic control unit (ECU) electrically connected to the one or more cameras and the user interface. The ECU may generate a composite image of the surrounding area of the vehicle based on the one or more fields of view included in the real-time video data. The ECU may further augment the composite image to include a virtual ramp extending from a virtual vehicle, the virtual vehicle being a virtual representation of the vehicle. The ECU may further display, via the user interface, the augmented composite image.

In one aspect, the subject matter may be embodied in a method for providing a virtual ramp preview to a driver of a vehicle having a deployable wheelchair ramp. The method may include generating, via an electronic control unit (ECU), one or more composite images of a surrounding area of the vehicle based on video data from one or more cameras. The method may further include displaying, via a user interface coupled to the ECU, a composite image of the one or more composite images. The method may further include augmenting, via the ECU, the displayed composite image to include a virtual ramp to assist a driver with estimating an amount of space that is required to deploy the deployable wheelchair ramp.

Disclosed herein are methods, systems, devices, and/or vehicles for implementing a virtual ramp display and preview system. When a driver is in a vehicle that has a deployable (e.g., handicap) ramp, the driver may need assistance with determining how and where the ramp will be deployed and if there are any obstacles or safety concerns in the way of the ramp. This advantageously assists the driver and/or passengers to determine the proper place for the vehicle to be parked or stopped so the ramp can be properly and safely deployed and to avoid having to repeatedly move and/or reposition the vehicle. The virtual ramp preview system allows for an augmented real-time composite image of a surrounding area of the vehicle that is displayed on a user interface (e.g., a mobile device, a head unit or a display screen) of the vehicle with a virtual (i.e., computer graphics or actual photo or digital overlay) rendering of a ramp (e.g., a deployed wheelchair ramp of the vehicle). The real-time composite image may be of a viewpoint selected from one or more viewpoints. The virtual ramp preview system may generate the one or more viewpoints by combining one or more fields of view of the surrounding area captured by one or more cameras of the vehicle (e.g., a one, two, four, and/or six camera based panoramic view monitor (PVM) system of the vehicle). The virtual ramp may provide a visual of the ramp deployed and/or a visual estimation of the amount of space that is required to deploy the ramp of the vehicle by showing a virtual representation of the ramp in a deployed state (i.e., extending out from the vehicle), with the virtual ramp accurately depicting the length and location of the ramp in the deployed state (i.e., the virtual ramp is displayed as extending out from a location on the vehicle that corresponds to where the ramp would extend out of if deployed). This advantageously allows the driver and/or passengers to properly park or stop the vehicle so the ramp can be safely and properly deployed in the correct location. Particular embodiments of the subject matter described in this disclosure may be implemented to realize one or more of the following advantages.

The virtual ramp preview system may allow a driver to assess whether there is sufficient space and acceptable and safe conditions (e.g., absence of bumps, obstacles, puddles, uneven surfaces, etc.) to deploy the ramp and/or for a wheelchair user to use the ramp at a particular location. The driver may make the assessment without having to physically deploy the ramp or step outside the vehicle to look at the surrounding conditions.

Moreover, the virtual ramp display and preview system may combine the one or more fields of view captured by the one or more cameras into the one or more viewpoints to assist the driver in finding an acceptable area to park or stop and deploy the ramp, with the driver having the ability to select which viewpoint is displayed by the user interface of the vehicle. For example, the virtual ramp preview system may provide a panoramic bird's eye viewpoint of the vehicle and the surrounding area of the vehicle and/or a viewpoint from a side of the vehicle having the ramp. The virtual ramp preview system may also provide a virtual three-dimensional (3D) perspective viewpoint of the side of the vehicle having the ramp. For example, the virtual 3D perspective viewpoint may be a virtual rendering of a rear or front of the vehicle as seen from a viewpoint that is 1-25 feet behind or in front of the vehicle and/or 1-10 feet to the side of the vehicle.

Moreover, the virtual ramp preview system may use the one or more cameras and monocular depth estimation and/or may include one or more distance sensors to provide one or more height overlays, one or more distance overlays, and/or one or more warning overlays on the real-time composite image to further assist the driver. For example, the virtual ramp preview system may determine a height of one or more objects within the surrounding area of the vehicle, a distance from a bottom of the virtual ramp to the one or more objects, and/or a distance between the vehicle and the one or more objects. The virtual ramp preview system may then augment the real-time composite image to display the one or more height overlays corresponding to the height of the one or more objects and/or the one or more distance overlays corresponding to the distance between the vehicle and the one or more objects. The virtual ramp preview system may further augment the real-time composite image to display the one or more warning overlays when the virtual ramp preview system determines that the ramp would contact the one or more objects if deployed and/or if the ground underneath the virtual ramp is uneven due to the presence of an object and/or a hole.

is a block diagram for an example ramp preview system(also can be referred to as a virtual ramp preview system). The ramp preview systemor a portion thereof may be retrofitted, coupled to, include, or be included within a vehicleor separate from the vehicle. The vehiclemay be a conveyance capable of transporting a person, an object, or a permanently or temporarily affixed apparatus. The vehiclemay be a self-propelled wheeled conveyance, such as a car, a sports utility vehicle, a truck, a bus, a van or other motor, battery or fuel cell driven vehicle. For example, the vehiclemay be an electric vehicle, a hybrid vehicle, a hydrogen fuel cell vehicle, a plug-in hybrid vehicle or any other type of vehicle that has a fuel cell stack, a motor, an engine, and/or a generator. Other examples of vehicles include bicycles, trains, planes, or boats, and any other form of conveyance that is capable of transportation. The vehiclemay be semi-autonomous or autonomous. That is, the vehiclemay be self-maneuvering and navigate without human input. An autonomous vehicle may have and use one or more sensors and/or a navigation unit to drive autonomously.

The ramp preview systemand/or the vehiclemay include a motor and/or generatorand a battery. The motor and/or generatormay be located within an engine bay of the vehicle. The motor and/or generatormay be an internal combustion engine (ICE). In this regard, the motor and/or generatormay combust an air and fuel mixture to provide power to the vehicleand/or components of the vehicleand/or the ramp preview system. Accordingly, the motor and/or generatorcan cause the vehicleto accelerate, decelerate, or maintain a desired velocity. It should be understood that the motor and/or generatormay include combinations of an ICE and an electric motor, such as for hybrid vehicle applications for example. In examples, the motor and/or generatormay be an electric motor. In this regard, the motor and/or generatormay be an electric motor and an electric generator that converts electrical energy into mechanical power, such as torque, and converts mechanical power into electrical energy. The motor and/or generatormay be electrically connected to the battery. The motor and/or generatormay convert energy from the batteryinto mechanical power, and may provide energy back to the battery, for example, via regenerative braking. The batterymay be electrically connected to the motor and/or generatorand may provide electrical energy to and/or receive electrical energy from the motor and/or generator. The batterymay provide electrical energy to the ramp preview system.

is an illustration of an example vehicleincluding the example ramp preview systemof. With combined reference to, the ramp preview systemmay further include a rampcoupled to the vehicleand/or a floor of the vehicle. The rampmay be a deployable ramp and/or a deployable wheelchair ramp and may have a deployed (or extended) state (or position)and a retracted (or stowed) state (or position). The rampmay be a side entry ramp (i.e., the rampis located on and deployable from a left side or a right side of the vehicle) and/or a rear entry ramp (i.e., the rampis located on and deployable from a rear of the vehicle). The rampmay be configured to deploy from the vehiclethereby extending outwardly from the right side, the left side, and/or the rear of the vehicleto form a ramp and/or a bridge between a cabin of the vehicleand the ground outside of the vehicle. In examples, the rampmay be an infloor (or slide out) ramp (i.e., the rampdeploys from and retracts into the floor of the vehiclesuch that when in the retracted state, the rampis located and/or stowed within the floor of the vehicle). In examples, the rampmay be a foldout ramp (i.e., the rampdeploys from and retracts onto the floor of the vehiclesuch that when in the retracted state, the rampis located and/or stowed atop the floor of the vehicle). The rampmay extend to the deployed stateby unfolding and/or sliding out from the vehiclemanually and/or automatically. The rampmay extend out a predetermined distancefrom the vehicle. The predetermined distancemay be based on a length of the ramp. In examples, the rampmay extend out at a variable distance from the vehiclebased on settings set by the driver of the vehicle.

The ramp preview systemmay further include one or more cameras. The one or more camerasmay be coupled to an exteriorof the vehicleand/or an interior of the vehiclesuch that the one or more cameraslook out toward a surrounding areaof the vehicle. The one or more camerasmay include a front camera, a left side camera, a right side camera, and/or a rear camera. The one or more camerasmay be an analog camera, a digital camera, an infrared thermal camera, and/or a night vision camera (e.g., utilizing active illumination and/or image intensification). The one or more camerasmay provide, capture, and/or record images and/or real-time video of the surrounding areaof the vehicle. The one or more camerasmay also have pattern recognition capabilities to view the surrounding areaand identify people and/or obstacles such as bumps, curbs, puddles, uneven surfaces, potholes, etc. to avoid as a possible location for the ramp deployment and to identify safe and good locations for deployment of the ramp. The image of the ramp may be displayed in red when the deployment location is not ideal and in green when the deployment location is ideal.

The surrounding areamay be a portion of a surrounding environment of the vehiclethat is visible to the one or more camerasand/or displayed to the driver (e.g., via a user interfaceof the vehicle). Each camera of the one or more camerasmay cover a field of view of one or more fields of view. The one or more fields of view may correspond to a panoramic (i.e., 360-degree) view around the vehiclewhen the one or more fields of view are combined to form one or more viewpoints. In examples, the one or more fields of view may correspond to a less than 360-degree view (e.g., a 180-degree view) when the one or more fields of view are combined to form the one or more viewpoints.

The front cameramay be coupled to the exteriorand/or a front bumperof the vehicle. In examples, the ramp preview systemmay include a plurality of front cameras each having a different field of view and/or orientation. The front cameramay provide, capture, and/or record real-time video of a front field of view (e.g., capturing a portion of the surrounding areaof the vehiclethat is in front of the vehicle).

The left side cameramay be coupled to the exteriorand/or a left side of the vehicle(e.g., a left side mirror, a left side quarter panel, a left side door, etc.). In examples, the ramp preview systemmay include a plurality of left side cameras each having a different field of view and/or orientation. The left side cameramay provide, capture, and/or record real-time video of a left field of view (e.g., capturing a portion of the surrounding areaof the vehiclethat is to the left of the vehicle).

The right side cameramay be coupled to the exteriorand/or a right side of the vehicle(e.g., a right side mirror, a right side quarter panel, a right side door, etc.). In examples, the ramp preview systemmay include a plurality of right side cameras each having a different field of view and/or orientation. The right side cameramay provide, capture, and/or record real-time video of a right field of view (e.g., capturing a portion of the surrounding areaof the vehiclethat is to the right of the vehicle).

The rear cameramay be coupled to the exterior, a rear bumper, a tailgate, and/or a liftgate of the vehicle. In examples, the ramp preview systemmay include a plurality of rear cameras each having a different field of view and/or orientation. The rear cameramay provide, capture, and/or record real-time video of a rear field of view (e.g., capturing a portion of the surrounding areaof the vehiclethat is to the rear of the vehicle).

The ramp preview systemmay further include one or more sensors (or one or more distance sensors). The one or more sensorsmay be a camera, a radar sensor, a sonar sensor, a lidar sensor, and/or another distance measuring sensor. The one or more sensorsmay measure, detect, and/or determine spatial information of one or more objectsin the surrounding areaof the vehicleby periodically and/or constantly scanning (or monitoring) the surrounding areaand constructing a point cloud. The spatial information may include a width, a height, and/or a depth of the one or more objectsand a position and/or location of the one or more objectsrelative to the vehicle. The one or more objectsmay include the ground outside of the vehicle, a curb, a sidewalk, a railing, a pole, a tree, a wall, a pile of snow, and/or any other stationary object. The one or more sensorsmay include a front distance sensor, a left side distance sensor, a right side distance sensor, and/or a rear distance sensor. In examples, the one or more objectsmay include one or more forward objects, one or more left side objects, one or more right side objects, and/or one or more rear objects. In examples where the one or more sensorsinclude a camera, the one or more sensorsmay be the same as the one or more camerasor may be in addition to the one or more cameras. The ramp preview systemmay use monocular and/or binocular depth estimation to determine the spatial information of the one or more objectswhen the one or more sensorsinclude one or more cameras.

The front distance sensormay be coupled to the exteriorand/or the front bumperof the vehicle. The front distance sensormay measure, detect, and/or determine spatial information of the one or more forward objects in the surrounding areaof the vehiclethat are in front of the vehicle. In examples, the ramp preview systemmay include a plurality of front distance sensors.

The left side distance sensormay be coupled to the exteriorand/or the left side of the vehicle(e.g., the left side mirror, the left side quarter panel, the left side door, etc.). The left side distance sensormay measure, detect, and/or determine spatial information of the one or more left side objects in the surrounding areaof the vehiclethat are to the left of the vehicle. In examples, the ramp preview systemmay include a plurality of left side distance sensors.

The right side distance sensormay be coupled to the exteriorand/or the right side of the vehicle(e.g., the right side mirror, the right side quarter panel, the right side door, etc.). The right side distance sensormay measure, detect, and/or determine spatial information of the one or more right side objects in the surrounding areaof the vehiclethat are to the right of the vehicle. In examples, the ramp preview systemmay include a plurality of right side distance sensors.

The rear distance sensormay be coupled to the exteriorand/or the rear bumper of the vehicle. The rear distance sensormay measure, detect, and/or determine spatial information of the one or more rear objects in the surrounding areaof the vehiclethat are to the rear of the vehicle. In examples, the ramp preview systemmay include a plurality of rear distance sensors.

The ramp preview systemmay further include one or more processors, such as an electronic control unit (ECU). The ECUmay be implemented as a single ECU or in multiple ECUs. The ECUmay be electrically connected to some or all of the components of the vehicleand/or the ramp preview system. The ECUmay be electrically connected to the ramp, the one or more cameras, the one or more sensors, a memory, a user interface, a network access device, a navigation unit, and/or a user input device. The ECUmay include one or more processors (or controllers) specifically designed for controlling operations of the vehicle, such as accelerating, braking, controlling the rampto deploy and/or retract, controlling a panoramic view monitor (PVM) of the vehicle(e.g., the one or more cameras), etc.

The ramp preview systemmay further include the memory. The memorymay be electrically connected to the ECU. In examples, the memorymay be communicatively coupled (e.g., via network) to the ECUsuch that the memoryis remote from the ECUand/or the vehicle. In other examples, the memorymay be electrically connected to the ECUand a remote memory (e.g., a remote database) may be communicatively coupled to the ECU, with the remote memory having similar, additional, and/or different functions as the memory(e.g., greater storage capacity, enabling over-the-air updates, etc.). The memorymay store instructions to execute on the ECUand may include one or more of a random access memory (RAM) or other volatile or non-volatile memory. The memorymay be a non-transitory memory or a data storage device, such as a hard disk drive, a solid-state disk drive, a hybrid disk drive, or other appropriate data storage, and may further store machine-readable instructions, which may be loaded and executed by the ECU. The memorymay store vehicle parameters (e.g., vehicle weight, vehicle length, vehicle width, vehicle height, transmission gear information, etc.).

The memorymay further store ramp parameters. The ramp parameters may include physical measurements and/or attributes of the rampincluding a width, a length (e.g., the length that the rampextends out from the vehiclewhen in the deployed statesuch as the predetermined distance), deployment location information, and/or clearance information. The deployment location information may include measurements corresponding to a location on the vehiclefrom where the rampdeploys. For example, the deployment location information may include which side of the vehiclethe rampdeploys from, and at what point (or area) of the vehiclethe rampdeploys from (e.g., a point or area with respect to the length, height, and width of the vehicle). The clearance information may correspond to dimensions of an area between a bottom of the rampand the ground when the rampis in the deployed state. The clearance information may include a height between the bottom of the ramp(in the deployed state) and the ground beneath the rampat various points along the ramp. The clearance information may include one or more heights because the rampmay be at an angle when deployed and not parallel to the ground. The clearance information may correspond to a maximum height of an object of the one or more objectsthat the rampmay deploy over such that the object would be beneath the rampand would not obstruct that ramp. The ramp parameters may be determined and/or set during manufacture of the vehicleand/or may be determined, set, and/or changed after the rampis manufactured and/or installed in the vehicle.

The ramp preview systemmay further include the user interface. The user interfacemay be located within the cabin of the vehicleand/or may be coupled to the dashboard of the vehicle. The user interfacemay provide an interface to a user of the vehicleto interact with and/or receive output from the ECU. The user interfacemay have a user interface element, such as a screen and/or a touchscreen with a button, a switch, a microphone, a speaker, a gesture monitoring sensor, a knob, a graphical user interface (GUI), and/or other input/output devices electrically connected to the ECUto provide input and/or output of information (or data) to and/or from the ECU. The user interfacemay include one or more displays to display a real-time composite image including one or more viewpoints of the surrounding areato a driver of the vehicleto assist the driver in finding an appropriate location to park and/or deploy the rampof the vehicle.

The ECUmay include one or more microcontrollers and/or image processing programs to generate and display, via the user interface, the real-time composite image including a viewpoint of the one or more viewpoints to the driver of the vehicle. To generate the one or more viewpoints, the ECUmay receive video signals (or video data) from the one or more camerasand may process the video signals (e.g., the video signals may be fed into an image processing program of the ECU). For example, the ECUand/or the image processing program may process the video signals by stitching together the one or more fields of view of the surrounding area(e.g., the front field of view, the left field of view, the right field of view, and/or the rear field of view) included within the video signals to create the one or more viewpoints. In examples, the video signals are processed based on a user selected (e.g., via the user interface) viewpoint of the one or more viewpoints. For example, the one or more fields of view may be combined to provide a panoramic bird's eye viewpoint of the vehicleand the surrounding areawith the vehiclebeing centered and/or offset within the panoramic bird's eye viewpoint. In examples, the one or more fields of view may be combined to provide one or more 3D perspective viewpoints of the surrounding area. In further examples, the one or more fields of view may be combined to provide a ramp side viewpoint of the surrounding area. The ECUmay continually generate the one or more viewpoints such that the one or more viewpoints may each provide a real-time view of the surrounding areaas the vehiclemoves through the surrounding environment.

In examples, each viewpoint of the one or more viewpoints generated by the ECUmay include perspective transformation. Perspective transformation may provide the appearance that the one or more viewpoints, when displayed on the user interface, are being captured from a “camera” that is hovering above, behind, and/or to a side of the vehicle. In examples, a user may select (e.g., via the user interface) a dynamic viewpoint that enables the user to drag and/or move the “camera” to change the viewpoint being displayed. For example, the dynamic viewpoint may enable the user to freely drag and/or move (e.g., via a touchscreen and/or a touchpad of the user interface) a displayed viewpoint to view the surrounding areafrom a plurality of angles. Thus, the dynamic viewpoint may enable the user to view the surrounding areafrom other angles in addition to the panoramic bird's eye viewpoint, the one or more 3D perspective viewpoints, and/or the ramp side viewpoint. In various aspects, the ECUmay receive the images/viewpoints from the one or more camerasand using image and pattern recognition technology, may identify and display the ramp as an image or overlay and identify and notify and/or direct the driver of a safe and acceptable location for deployment.

are illustrations of example composite images (or real-time composite images)-each including a viewpoint of the one or more viewpoints that may be displayed by the user interface. The ramp preview systemmay display a first composite imagehaving a panoramic bird's eye viewpoint (as shown by). The ramp preview systemmay further display a second composite imagehaving a rear 3D perspective viewpoint (as shown by). The ramp preview systemmay further display a third composite imagehaving a front 3D perspective viewpoint (as shown by). The ramp preview systemmay further display a fourth composite imagehaving a ramp side viewpoint (as shown by). With combined reference to, the composite images-may each include a virtual rampextending from a virtual representation of the vehicle(i.e., a virtual vehicle, a virtual vehicle, or a virtual vehicle, respectively) and the fourth composite imagemay include the virtual rampextending from the viewpoint (i.e., having the appearance of extending forward and/or out from a side of a virtual vehicle) to assist the driver in finding an appropriate location to park and/or deploy the rampof the vehicle.

The virtual rampmay be a computer graphics rendering of the rampand may be scaled based on the dimensions of the ramp. The virtual rampmay appear in the composite images-as extending outward from the virtual vehicle,,at a location on the vehiclewhere the rampmay deploy from the vehicle. In the fourth composite image, the virtual rampmay appear as extending outward from the viewpoint and in some examples may appear as extending outward from a side of a virtual vehicle. The virtual rampmay be positioned and/or sized according to known reference points in the surrounding areaand/or on the vehiclesuch that the virtual rampaccurately represents the size and location of the rampin the deployed statewithin the composite images-. The known reference points may be determined via the one or more sensors. The ECUmay use augmented reality processing techniques to overlay the virtual ramponto the real-world video of the composite images-. In examples, the ECUand/or the user interfacemay generate (or render) the virtual rampfrom a 2D or 3D model saved in the memory. The virtual rampmay allow the driver to visually estimate a distance between the virtual rampand one or more objects within the surrounding area, with the distance between the virtual rampand the one or more objects corresponding to a distance between the rampand the one or more objects if the rampwere deployed. The one or more objects may be captured by the one or more camerasand displayed within the composite images-and/or may be virtual representations of the one or more objects generated from data collected by the one or more camerasand/or the one or more sensors. Combining real-world video from the one or more camerasand spatial information from the one or more sensorshelps to ensure that the ramp preview systemdetects and/or displays every object within the surrounding area.

In examples, the ECUmay annotate the composite images-with distance information corresponding to a distance between the vehicleand/or the virtual rampand one or more objects within the surrounding area. The distance information may appear as an overlay (e.g., a computer graphics rendering) positioned by the one or more objects in the composite images-. For example, the one or more objects within the surrounding areamay include a fenceand a pole. The composite images-may include a distance annotationthat displays the distance between, for example, the fenceand the vehicle(e.g., 3 feet). The distance annotationmay include a number and a unit of measurement (e.g., “3 feet”) and/or may include a notation specifying the number relates to a distance (e.g., “3 feet away”). In examples, when the distance between the one or more objects and the vehicleis less than the length of the ramp(e.g., the distance between the poleand the vehicle), the one or more objects (e.g., the pole) and/or the virtual rampmay be highlighted in red (represented by diagonal lines across the polein) and/or may include a warning annotationby the one or more objects to warn the driver that the rampwill contact the one or more objects if the rampwere deployed in the current location of the vehicle.

The ramp preview systemmay determine the distance between the one or more objects and the vehiclebased on the spatial information received from the one or more sensors. The one or more sensorsand/or the ECUmay construct a point cloud of the surrounding areaand/or the one or more objects within the surrounding areato determine the dimensions of the one or more objects and the spatial relationship of the one or more objects relative to the vehicle. Moreover, the ramp preview systemmay determine whether the one or more objects would contact the rampif the rampwere deployed in the current location of the vehiclebased on the spatial information received from the one or more sensorsand the ramp parameters stored in the memory. For example, the ECUmay determine that the poleis X feet away from the vehicleand may then compare X to a length Y of the rampin the deployed state. If X is equal to or less than Y, the ECUmay annotate the composite images-with a warning annotationby the pole. In examples, the ECUmay annotate the composite images-with the warning annotationby the poleif, in addition to X being equal to or less than Y, the ECUfurther determines that the rampwould contact the polebased on the width of the rampand the current location of the vehicle(i.e., the poleis in front of a deployment location of the ramp). In examples, the ECUmay include a buffer zone that is added onto the length and/or the width of the rampto ensure that there is enough room between the rampand the one or more objects for a wheelchair user to use the ramp.

In examples, the ECUmay further annotate the composite images-with height information corresponding to a height of the one or more objects within the surrounding areaand/or a distance between a top of the one or more objects and the bottom of the virtual ramp. The height information may appear as an overlay (e.g., a computer graphics rendering) positioned by the one or more objects in the composite images-. For example, the one or more objects within the surrounding areamay include a rockand a pothole. The composite images-may include a height annotationthat displays the height of, for example, the rockrelative to the ground and/or a streetbelow the rock(e.g., 5 inches). The height annotationmay include a number and a unit of measurement (e.g., “5 inches”) and/or may include a notation specifying the number relates to a height (e.g., “5 inches tall”). The composite images-may further include a depth annotationthat displays a depth of, for example, the potholerelative to the ground and/or the street(e.g., −5 inches). In examples, when the height of an object of the one or more objects is too great for the rampto deploy over the object, the object and/or the virtual rampmay be highlighted in red and/or may include a warning annotationby the object to warn the driver that the rampwill contact the object if the rampwere to be deployed in the current location of the vehicle.

The ramp preview systemmay determine the height of the one or more objects based on the spatial information received from the one or more sensors. Moreover, the ramp preview systemmay determine whether the object would contact the rampif the rampwere to be deployed in the current location of the vehicleand over the object based on the spatial information received from the one or more sensorsand the ramp parameters stored in the memory, specifically the clearance information included in the ramp parameters. For example, the ECUmay determine whether the object will contact the rampif the rampwere deployed over the object by comparing the height of the object to the height of a portion of the rampthat would be above the object if the rampwere deployed at the current location of the vehicle.

As shown by, the panoramic bird's eye viewpoint included in the first composite imagemay appear as a top view of the vehicleand the surrounding area. The panoramic bird's eye viewpoint may appear as a view from a viewpoint that is 1-25 feet above the vehicle. The first composite imagemay include the virtual vehicle, the virtual vehiclebeing a virtual representation of a top of the vehicle.

In examples, the ECUmay further annotate the composite images-with ramp information corresponding to a length and/or a width of the rampas shown by. The ramp information may further assist a driver with determining how much space is required to deploy the ramp. The ramp information may appear as an overlay (e.g., a computer graphics rendering) positioned by and/or on the virtual rampin the composite images-. The composite images-may include a ramp annotationthat displays the length of the rampin the deployed state(e.g., 3 feet 6 inches). The ramp annotationmay include a number and a unit of measurement (e.g., “3 feet 6 inches”) and/or may include a notation specifying the number relates to a length (e.g., “3 feet 6 inches long”).

As shown by, the rear 3D perspective viewpoint included in the second composite imagemay appear as a rear to front view of the vehicleand the surrounding areaat a ground level. The rear 3D perspective viewpoint may appear as a view from a viewpoint that is 1-25 feet behind and/or 1-10 feet to a side of the vehicle. When a hole is within the surrounding area, such as the pothole, the second composite imagemay include a virtual representation of the ground below the streetto better illustrate to the driver the depth of the hole and/or the unevenness of the street. The second composite imagemay include the virtual vehicle, the virtual vehiclebeing a virtual representation of the rear of the vehicle.

As shown by, the front 3D perspective viewpoint included in the third composite imagemay appear as a front to rear view of the vehicleand the surrounding areaat a ground level. The front 3D perspective viewpoint may appear as a view from a viewpoint that is 1-25 feet in front of and/or 1-10 feet to a side of the vehicle. When a hole is within the surrounding areathe third composite imagemay include a virtual representation of the ground below the streetto better illustrate to the driver the depth of the hole and/or the unevenness of the street. The third composite imagemay include the virtual vehicle, the virtual vehiclebeing a virtual representation of the front of the vehicle.

As shown by, the ramp side viewpoint included in the fourth composite imagemay appear as an outward view of the surrounding areafrom a side of the vehiclewith the virtual rampextending out and being centered in the ramp side viewpoint (in examples, the virtual rampmay be offset to the left or the right in the ramp side viewpoint). The ramp side viewpoint may appear as a view from a viewpoint that is on the exteriorof the vehicleand looking outwardly from a left side or a right side of the vehicle. In examples, the ramp side viewpoint may be from the rear of the vehicle. The fourth composite imagemay include a virtual vehicle, the virtual vehicle being a virtual representation of a ramp side of the vehicle.

In examples, the ECUmay further annotate the composite images-with parking assessment information corresponding to an assessment of a parking location (i.e., a current location of the vehicle) as shown by. In examples, the ramp preview systemmay utilize artificial intelligence (AI) (e.g., generative AI) to provide the parking assessment information in real time. The parking assessment information may appear as an overlay (e.g., a computer graphics rendering) in the composite images-(e.g., near the top, bottom, and/or side of the composite images-). For example, the composite images-may include a negative parking assessment annotationthat displays a message stating, for example, “Insufficient space to use the ramp” or “A better location to use the ramp is recommended.” The negative parking assessment annotationmay be generated and included within the composite images-when the composite images-include a warning annotation (e.g., warning annotation) and/or when the ramp preview systemdetermines (e.g., via the AI) a parking location is not suitable to deploy the ramp. The AI may be a machine learning system (or program) that is trained on a data set including measurements and/or visual data of suitable and unsuitable parking locations. The machine learning system may compare spatial information from the one or more sensorsand/or real-time video data from the one or more cameraswith the measurements and/or the visual data of the suitable and the unsuitable parking locations to determine whether a parking location is suitable. A positive parking assessment annotation may be generated when the composite images-do not include a warning annotation (e.g., warning annotation) and/or when the ramp preview systemdetermines (e.g., via the AI) a parking location is suitable to deploy the ramp. The positive parking assessment annotation may state, for example, “This is a good location to use the ramp.”

The ramp preview systemmay further include the network access device. The network access devicemay be electrically connected to the ECUand may include a communication port or channel, such as one or more of a Wi-Fi unit, a Bluetooth® unit, a Radio Frequency Identification (RFID) tag or reader, a DSRC unit, and/or a cellular network unit for accessing a network(e.g., CDMA, GSM, 3G, 4G, 5G, etc.). The network access devicemay transmit data to and receive data from devices and systems not directly connected to the vehicle. For example, the ECUmay communicate with a user device(e.g., a key fob, a mobile device, phone, tablet, laptop, etc.) through the network access device. In examples, a user (e.g., the owner of the vehicle) may use the user deviceto communicate with the ECUto start the vehicle, activate the ramp preview system, and/or view the composite images-of the ramp preview system.

In examples, the network access devicemay enable the ECUto receive additional fields of view of the surrounding areafrom one or more remote cameras. The one or more remote camerasmay be cameras of other vehicles and/or may be other third-party cameras (e.g., security cameras, parking lot cameras, etc.) that are available to provide the additional fields of view of the surrounding area. In examples, the one or more remote camerasmay improve the one or more composite images-by providing fields of view that may not otherwise be available to the ECUdue to the location of the one or more camerason the vehicle.

The ramp preview systemmay further include the navigation unit. The navigation unitmay be electrically connected to the ECUand may provide vehicle information (or data) and/or navigational map information to the ECU. The navigation unitmay have and/or be connected to a Global Positioning System (GPS) device. The vehicle information may include a current position and/or location of the vehicle, a current time at the current position, a direction of travel, and/or a current speed of the vehicle. In examples, the ECUmay automatically start the ramp preview systembased on the vehicle information received from the navigation unit. For example, the ECUmay learn one or more locations where the ramp preview systemis regularly activated and may automatically activate the ramp preview systemwhen the vehicleapproaches and/or arrives at the one or more locations. In examples, the one or more locations may be stored in the memoryby a user of the vehicle(e.g., via the user interface).

The ramp preview systemmay further include a user input device. The user input devicemay be located within the cabin of the vehicleand/or may be coupled to the dashboard of the vehicle. The user input devicemay be electrically connected to the ECU. The user input devicemay be and/or include one or more buttons, one or more switches, one or more dials, one or more touch screens, one or more gesture control sensors, one or more voice control sensors, and/or any other input devices. The user input devicemay receive and/or detect user input from a user to activate and/or deactivate the ramp preview system(e.g., start and/or stop displaying the composite images-on the user interface).

is a flow diagram of an example processfor controlling the ramp preview system. One or more computers or one or more data processing apparatuses, for example, the ECUof the ramp preview systemof, appropriately programmed, may implement the process. For ease of description, the processis described below with reference to. The processof the present disclosure, however, is not limited to use of the exemplary ramp preview systems of.

The ramp preview systemmay receive, via the one or more cameras, video data including the one or more fields of view of the surrounding areaof the vehicle().

The ramp preview systemmay generate one or more composite images-based on the video data (). The one or more composite images-may include the panoramic bird's eye viewpoint, the rear 3D perspective viewpoint, the front 3D perspective viewpoint, and/or the ramp side viewpoint. To generate the one or more composite images-, the ECUmay stitch together (or combine) the one or more fields of view included in the video data. In examples, a user may select a composite image of the one or more composite images-to be displayed on the user interface. The ECUmay stitch together the one or more fields of view based on a viewpoint included in the user selected composite image.

The ramp preview systemmay display, via the user interface, a composite image of the one or more composite images-(). The composite image may be selected by a user through the user interface. The displayed composite image may allow a driver to see one or more objects within the surrounding areaof the vehicle.