Initial location determination of uncrewed aerial vehicles from uncalibrated location observations

This application claims priority to U.S. provisional patent application No. 63/521,209, filed on Jun. 15, 2023, and titled “Initial Location Determination of Uncrewed Aerial Vehicles from Uncalibrated Location Observations,” which is hereby incorporated by reference as if fully set forth in this description.

An uncrewed vehicle, which may also be referred to as an autonomous vehicle, is a vehicle capable of travel without a physically-present human operator. An uncrewed vehicle may operate in a remote-control mode, in an autonomous mode, or in a partially autonomous mode. The term “unmanned” may sometimes be used instead of, or in addition to, “uncrewed,” and for the purposes of this disclosure, it should be understood that both terms have the same meaning, and may be used interchangeably.

When an uncrewed vehicle operates in a remote-control mode, a pilot or driver that is at a remote location can control the uncrewed vehicle via commands that are sent to the uncrewed vehicle via a wireless link. When the uncrewed vehicle operates in autonomous mode, the uncrewed vehicle typically moves based on pre-programmed navigation waypoints, dynamic automation systems, or a combination of these. Further, some uncrewed vehicles can operate in both a remote-control mode and an autonomous mode, and in some instances may do so simultaneously. For instance, a remote pilot or driver may wish to leave navigation to an autonomous system while manually performing another task, such as operating a mechanical system for picking up objects, as an example.

Various types of uncrewed vehicles exist for various different environments. For instance, uncrewed vehicles exist for operation in the air, on the ground, underwater, and in space. Examples include quad-copters and tail-sitter uncrewed aerial vehicles (UAVs), among others. Uncrewed vehicles also exist for hybrid operations in which multi-environment operation is possible. Examples of hybrid uncrewed vehicles include an amphibious craft that is capable of operation on land as well as on water or a floatplane that is capable of landing on water as well as on land. Other examples are also possible.

Examples disclosed herein include systems and methods involving automated location determination of aerial vehicles (UAVs) using observations obtained by the UAVs while operating in navigational state that may not be fully or even partially initialized.

In a first aspect, an uncrewed aerial vehicle (UAV) may comprise an imaging system and a control system. The control system may be configured to: cause the UAV to hover above a particular charging pad within a portion of a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; cause the imaging system capture an aerial image of the portion of the ground-based cluster, including the particular charging pad, while hovering above the particular charging pad; derive cluster-portion observation data from the aerial image, wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; send, via electronic transmission, the cluster-portion observation data to a computing system in an infrastructure support network for UAVs; and receive, from the computing system, via electronic transmission, location information, the location information indicating that UAV's geolocation is a geolocation of the particular charging pad.

In a second aspect, a computing system in an infrastructure support network for uncrewed aerial vehicles (UAVs) may comprise one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing system to carry out operations including: receiving, from a UAV, via electronic transmission, cluster-portion observation data derived from an aerial image of a portion of a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; wherein the aerial image was captured by the UAV while hovering above a particular charging pad within the portion of the ground-based cluster, and wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; identifying at least one mapped fiducial marker in a stored reference map of the ground-based cluster that matches one of the one or more fiducial markers in the cluster-portion observation data; identifying a mapped charging pad in the reference map as a match to the particular charging pad in the cluster-portion observation data, based on the identified at least one mapped fiducial marker and its position in the reference map relative to the identified mapped charging pad; determining a geolocation and an orientation of the particular charging pad according to a recorded geolocation and orientation for the identified mapped charging pad; and sending, via electronic transmission, location information to the UAV, the location information indicating that the UAV's geolocation is the determined geolocation of the particular charging pad.

In a third aspect, a system may comprise a server in an infrastructure support network for uncrewed aerial vehicles (UAVs); a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; and a UAV comprising an imaging system, and a control system. The UAV control system may be configured to: cause the UAV to hover above a particular charging pad within a portion of the ground-based cluster; cause the imaging system capture an aerial image of the portion of the ground-based cluster, including the particular charging pad, while hovering above the particular charging pad; derive cluster-portion observation data from the aerial image, wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; send, via electronic transmission, the cluster-portion observation data to the server; and receive, from the server, via electronic transmission, location information, the location information indicating that UAV's geolocation is a geolocation of the particular charging pad. The server may be configured to: receive, from the UAV, the cluster-portion observation data; identify at least one mapped fiducial marker in a stored reference map of the ground-based cluster that matches one of the one or more fiducial markers in the cluster-portion observation data; identify a mapped charging pad in the reference map as a match to the particular charging pad in the cluster-portion observation data, based on the identified at least one mapped fiducial marker and its position in the reference map relative to the identified mapped charging pad; determine the geolocation and an orientation of the particular charging pad according to a recorded geolocation and orientation for the identified mapped charging pad; and send the location information to the UAV.

In a fourth aspect, a method may be carried out by an uncrewed aerial vehicle (UAV). The method may comprise: causing the UAV to hover above a particular charging pad within a portion of a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; causing an imaging system of the UAV to capture an aerial image of the portion of the ground-based cluster, including the particular charging pad, while hovering above the particular charging pad; deriving cluster-portion observation data from the aerial image, wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; sending, via electronic transmission, the cluster-portion observation data to a computing system in an infrastructure support network for UAVs; and receiving, from the computing system, via electronic transmission, location information, the location information indicating that UAV's geolocation is a geolocation of the particular charging pad

In a fifth aspect, a method may be carried out by a computing system in an infrastructure support network for uncrewed aerial vehicles (UAVs). The method may comprise: receiving, from a UAV, via electronic transmission, cluster-portion observation data derived from an aerial image of a portion of a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; wherein the aerial image was captured by the UAV while hovering above a particular charging pad within the portion of the ground-based cluster, and wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; identifying at least one mapped fiducial marker in a stored reference map of the ground-based cluster that matches one of the one or more fiducial markers in the cluster-portion observation data; identifying a mapped charging pad in the reference map as a match to the particular charging pad in the cluster-portion observation data, based on the identified at least one mapped fiducial marker and its position in the reference map relative to the identified mapped charging pad; determining a geolocation and orientation of the particular charging pad according to a recorded geolocation and orientation for the identified mapped charging pad; and sending, via electronic transmission, location information to the UAV, the location information indicating that the UAV's geolocation is the determined geolocation of the particular charging pad.

In a sixth aspect, a method may be carried out by an uncrewed aerial vehicle (UAV) and a server in an infrastructure support network for UAVs. The method may comprise: the uncrewed aerial vehicle (UAV) hovering above a particular charging pad within a portion of a ground-based cluster of charging pads for UAVs, the cluster comprising the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout; the UAV capturing an aerial image of the portion of the ground-based cluster, including the particular charging pad, while hovering above the particular charging pad; the UAV deriving cluster-portion observation data from the aerial image, wherein the cluster-portion observation data comprise information indicating a position of the particular charging pad, and positions of one or more fiducial markers within the portion of the ground-based cluster relative to the particular charging pad; the UAV sending, via electronic transmission, the cluster-portion observation data to the server. The method may further comprise: the server receiving, from the UAV, the cluster-portion observation data; the server identifying at least one mapped fiducial marker in a stored reference map of the ground-based cluster that matches one of the one or more fiducial markers in the cluster-portion observation data; the server identifying a mapped charging pad in the reference map as a match to the particular charging pad in the cluster-portion observation data, based on the identified at least one mapped fiducial marker and its position in the reference map relative to the identified mapped charging pad; the server determining a geolocation and an orientation of the particular charging pad according to a recorded geolocation and orientation for the identified mapped charging pad; the server sending, via electronic transmission, location information to the UAV, the location information indicating that the UAV's geolocation is the determined geolocation of the particular charging pad. The method may still further comprise the UAV receiving, from the server, the location information.

In a seventh aspect, a non-transitory computer-readable medium may store instructions that, when executed by one or more processors of an uncrewed aerial vehicle (UAV), cause the UAV to perform operations corresponding to the method carried by a UAV, as described above.

In an eighth aspect, a non-transitory computer-readable medium may store instructions that, when executed by one or more processors of a computing system in an infrastructure support network for uncrewed aerial vehicles (UAVs), cause the system to perform operations corresponding to the method carried out by a computing system, as described above.

In a ninth aspect, a non-transitory computer-readable medium may store instructions that, when executed by one or more processors of a system comprising (i) a server in an infrastructure support network for uncrewed aerial vehicles (UAVs) and (ii) a UAV, cause the system to perform operations corresponding to the method carried out by a UAV and a server, as described above.

These, as well as other aspects, advantages, and alternatives, will become apparent to those of ordinary skill in the art by reading the following detailed description with reference where appropriate to the accompanying drawings. Further, it should be understood that the description provided in this summary section and elsewhere in this document is intended to illustrate the claimed subject matter by way of example and not by way of limitation.

Exemplary methods and systems are described herein. It should be understood that the word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation or feature described herein as “exemplary” or “illustrative” is not necessarily to be construed as preferred or advantageous over other implementations or features. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. The example implementations described herein are not meant to be limiting. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.

An example usage of UAVs may be to deliver various items to customers. For example, UAVs may be tasked with picking up a payload containing an item from a location and delivering the payload to a customer's residence, commercial building, or other location. In a typical usage scenario a group or fleet of UAVs may be deployed by a fleet operator to provide a delivery service for its own product offerings and/or for other vendors. A fleet of UAVs may be used for additional and/or other services, such as environment surveillance and emergency services support, among other non-limiting examples. The tasks associated with services provided by a fleet of UAVs may sometimes be referred to herein as “missions,” a term meant to convey that a set of operational steps that may be involved in carrying out a task, such as delivery.

In order to support a fleet of UAVs in their missions, as well as support the service or services provided by the fleet, a fleet operator may also deploy an infrastructure. As described in detail below, such an “infrastructure support network,” as it is sometimes referred to herein, may include servers and computing systems for managing mission operations, coordinating UAV tasks, interfacing with customers (or other service recipients), and maintaining fleet health and readiness. In accordance with example embodiments, the infrastructure may be implemented in a centralized and/or distributed fashion, and may be organized hierarchically. Infrastructure may also include and or be connected with one or more communications networks, such as a public and/or private internet, for supporting communications between other infrastructure elements.

One aspect of maintaining fleet health and readiness may entail charging batteries of individual UAVs. In accordance with example embodiments, battery charging may be implemented using large-scale deployments of charging pads arranged in groups or clusters located at various sites across an operating region, such as a city or metropolitan area. For example, each vendor using a UAV delivery service may host a cluster of 10-50 or more charging pads at each of its vendor sites. Each cluster may be powered by available AC (alternating current) power, and be relatively inexpensive to operate, incurring only electric utility fees for intermittent usage, for example. Charging pads may also serve as origination points and/or termination points of UAV missions and/or mission legs (or segments), such as delivery pick-up and/or drop-off points.

In accordance with example embodiments, charging pad clusters may be deployed as standalone sites and/or integrated with additional infrastructure elements or components for supporting other aspects of UAV fleet operation. For example, an aspect of mission operations may involve collecting and processing/analyzing various forms of operations reporting data from UAVs accumulated during missions, and providing various forms of operations configuration data to UAVs. Thus, infrastructure facilities for bulk data transfers between UAV and fleet support systems may include a wireless access point communicatively connected to a wireless and/or wireline broadband interface to a data backhaul network, such as a data/telecommunications carrier core network and/or internet. The data backhaul network may then be communicatively connected to servers and computer systems of the fleet support infrastructure. A UAV may communicate via a wireless connection with the wireless access point. In some instances, a UAV may have its own cellular wireless interface to a cellular data and telecommunications network.

Such “full-service” operations support facilities are sometimes referred to as “jet bridges,” in analogy to like-named facilities where commercial aircraft typically park at airport terminals. Thus, a jet bridge may provide one or more charging pads, as well as a communicative connection to an infrastructure support network. An infrastructure support network may include one or more servers that support flight scheduling, flight plans, and mission plans, for example. Jet bridges and/or standalone charging pad clusters may be deployed at warehouses, retail shopping centers, and/or other facilities that may be served by UAVs and/or support UAV operations.

A UAV may have a navigation system that provides a relatively coarse mode of navigational accuracy to enable the UAV to achieve, with a specified likelihood of navigational success, flying to within a threshold tolerance of a designated target location. The coarse mode of navigation may use GPS, for example, which could have limited and/or impeded accuracy in some circumstances. In an example usage scenario, the designated target location may be a designated target charging pad within a cluster of charging pads arranged in a grid or other layout, and the specified likelihood and threshold tolerance may be such that the UAV may have a non-vanishing likelihood of arriving at the wrong charging pad. The UAV navigation system may also provide a fine mode to augment its coarse navigational solution in order to be able to distinguish the correct, designated target charging pad from among other nearby pads that may be within the threshold tolerance. More particularly, in example embodiments a cluster of charging pads may include a distribution of fiducial markers at various positions and orientations relative to the charging pads that may be recognized by a UAV and used to achieve the requisite disambiguation.

In an example usage scenario, a UAV may have a mission that includes flying to and (possibly) landing on a designated charging pad in a cluster of charging pads. The UAV may be provided with a reference map of the cluster, including the fiducial markers. The reference map may represent a view of the cluster from above in which the spatial distribution of fiducial markers in the vicinity of any given charging pad form a sort of reference constellation or pattern that is distinguishable from similarly-formed reference constellations in the vicinities of other charging pads of the cluster. The UAV may thus use its coarse navigation mode to fly to and arrive at a coarse-navigation position within the threshold tolerance of the designated target charging pad, and then use an imaging system to acquire an image of the cluster from its perspective at the coarse-navigation position and orientation. The UAV may identify one or more observed charging pads in the image, as well as corresponding observed constellations of fiducial markers for each of the one or more observed charging pads. By applying one or another form a pattern matching operation to one or more of the observed constellations and one or more of the reference constellations in the reference map, the UAV may then distinguish or disambiguate between the identities or cluster positions of the one or more observed charging pads. In particular, the UAV may use the disambiguation to identify the designated target charging pad and/or disqualify a different observed charging pad that might otherwise be mis-identified as the designated target charging pad.

In some deployments, the charging pads of a cluster may be arranged in a layout, such as a regular grid or array. In accordance with example embodiments, by using constellations of fiducial markers at their respective relative positions and orientations with respect to charging pads of a cluster, the fiducial markers need not necessarily be deployed at precise geolocations and/or have precisely determined geolocations. In some examples, the distribution and orientations of fiducial markers at locations across a cluster may be random and/or regular. Additionally, a match between a reference constellation and an observed constellation need not necessarily be exact in order to confirm or disqualify an identification. Thus, this technique may tolerate some degree of discrepancy between a reference map of a cluster and the actual distribution of fiducial markers observed or captured in the UAV's image. Discrepancies could result from damage to, and/or debris on, one or more fiducial markers, for example. Some discrepancies may result, at least in part, simply from misalignment between observed and reference constellations. Straightforward geometric transformations (e.g., rotations and/or translations) may be applied to compensate for such misalignments (or other differences of perspective between observed and reference constellations).

Further, to the extent that the fiducial markers may be identified by visibly (or otherwise) recognizable physical features, all of the fiducial markers of a cluster need not necessarily have unique identifications in order for different constellations to be distinguishable. Thus, by using recognizable constellations of fiducial markers to disambiguate among charging pads, charging pad clusters including fiducial markers may be deployed without necessarily requiring determination of precise geolocations of pads or fiducial markers, or necessarily requiring unique identifications among fiducial markers of the cluster. As a resulting benefit, it may also be possible to reduce the need for on-site maintenance of fiducial markers compared to what might otherwise be required.

Because constellations or patterns of fiducial markers with respect to charging pads of a cluster derive from relative positions and orientations of the fiducial markers with respect to the charging pads, and do not necessarily depend on precise geolocations of the fiducial markers, the term “context-based navigation” may be used to describe the techniques and methodology of example embodiments herein. That is, recognizable patterns of fiducial markers may provide a context for positional and orientational awareness of a UAV with respect to specific charging pads of a cluster that does not necessarily require a determination of either the UAV's precise geolocation, or of the fiducials of a cluster.

As described, context-based navigation by UAVs, using observed relative positions of fiducial markers and charging pads, is supported in part by reference maps of charging clusters provided by a UAV ground-support system. Because charging pad clusters may be deployed without necessarily requiring determination of precise geolocations of pads or fiducial markers, or necessarily requiring unique identifications among fiducial markers of the cluster, generation of reference maps of charging pad cluster may not be able to rely on or assume availability of precise geolocations of ground-based assets (e.g., charging pads and fiducials). Accordingly, the inventors have devised techniques for generating reference maps that utilize aerial observations of apparent positions of assets in clusters by one or more UAVs. Advantageously, these techniques not only enable creation of reference maps well-suited for context-based navigation to be generated from specific survey, as well as routine aerial, observations by UAVs, without having or needing precise geolocation measurements of ground-based assets of clusters, but the techniques also allow routine monitoring of clusters for changes-both intentional and unintentional—in asset configurations. The routine monitoring may then be used to routinely update reference maps to keep them current for context-based navigation, and to detect configuration changes that may indicate cluster-site problems and/or require some form of on-site maintenance.

In accordance with example embodiments, reference maps of a cluster may be generated by a server or other processing system from aerial observations obtained from planned or incidental flights of UAVs over a locale of the cluster. For example, one or more UAVs may have missions that include acquiring aerial observations (e.g. images) of one or more clusters from various positions and perspectives above the clusters. UAVs may analyze image data to recognize charging pads and fiducials, and determine vector positions of the recognized assets with respect to the respective geolocations of the UAVs at the time of image capture. The UAVs may then construct aerial observation data including the vector positions and the measurements of the UAVs' positions and orientations, and transmit the aerial observation data to a server in an infrastructure support network for UAVs. The server (or servers) may use the observation data to construct a map graph of the cluster assets, and then apply an analytical optimization of the map graph in order generate a reference map of the cluster.

In accordance with example embodiments, reference map generation may be part of a system configured for tracking of ground-based assets of charging pad clusters. Non-limiting examples of ground-based assets include charging pad and fiducials of charging pad clusters of a UAV ground-support system. In further accordance with example embodiments, asset tracking may include an asset manager server for carrying out operations and computations of asset tracking and reference map generation and maintenance, and an associated asset database for storing reference maps of known clusters and records of known assets, among other possible information.

In further accordance with example embodiments, an asset manager may be configured to receive aerial observation data from one or more UAVs of one or more charging pad clusters. From time to time, the asset manager may thus use the received aerial observation data to generate a map of one or more clusters. When the asset manager generates a reference map for a previously known and recorded cluster, the newly-generated map and any existing map for the cluster may be compared. In this way, cluster maps may be updated as deemed necessary, and/or cluster-site changes of consequence may be detected or discovered. If a first-time map is generated for a new cluster, the new map may be added to the tracking system.

Also in accordance with example embodiments, generation of new and/or updated reference maps may yield identifications and positions of the assets contained in the maps. This information for each given asset may then be used to update a record for the given asset if it has been previously recorded, or create a new asset record if the given asset was not previously known or recorded.

In further accordance with example embodiments, updating maps and assets may include adjusting the position of an asset in a map, adding a new asset to an existing map, or deleting an asset from an existing map. Other forms of updates may be performed as well.

By having UAVs process/analyze image data and effectively distill it down to UAV aerial geolocations and vector positions of assets, aerial observation data may be rendered relatively small in size. Advantageously, this may allow UAVs to send aerial observation data to the server using relatively small cellular transmissions, and thus relatively low cost in bandwidth usage, transmission time, and monetary expense.

When a UAV first powers on, various of the UAV's operating systems and devices may need to undergo initializations and health and safety checks in order to bring the UAV into a flight-ready operational state. This may include verifying battery charge and flight-control systems, among other functions. In addition to operational readiness, a UAV's initial location and orientation may need to be known to some specified degree of accuracy in order to calibrate its navigation system and to support mission planning. But the UAV's geolocation and orientation may be indeterminate at power-on, except possibly for a general operational convention of the UAV initially residing within one of numerous ground-support facilities that include multiple take-off/landing platforms. In some examples, the take-off/landing platform may be a charging pad in a cluster of charging pads. Even with a presumption that a UAV powers on from a charging pad cluster, however, neither the UAV nor a UAV support system may necessarily know which of numerous charging pads of a given cluster the UAV is on, and/or even which of numerous clusters the UAV is at, when it powers on.

One technique for determining a UAV's geolocation at power-up is to use a GPS system. While this can work in principle, in practice GPS satellite signals received at ground level, where a UAV may be positioned when it powers on, may be subject to direct path and/or multipath interference, or other forms of signal degradation, that can reduce the accuracy of GPS-based geolocation determination below what may be needed for proper navigation system calibration and mission flight planning. GPS signal degradation may be mitigated or overcome by causing the UAV to ascend vertically from its initial power-up position to a sufficiently high altitude where there is little or no interference. For example, at approximately 20 m altitude above ground level (“AGL”) or higher, a UAV may be able to receive a clear enough and strong enough GPS signal to determine its geolocation to within approximately three meters accuracy. However, an aerial vehicle at 20 m ALG may be subject to aviation regulations or other flight or regulatory considerations that may incur costs and/or introduce operational complexities. Thus, it would be desirable to be able to determine the initial location of a UAV at power-up without needing to rely on an AGL that enables adequate GPS geolocation, but that might also trigger regulatory and/or operational considerations.

Accordingly, the inventors have devised techniques for using a low-altitude initial vertical flight of a UAV upon power-up to both initialize its operational state, and to acquire one or more images of the UAV's launch point and a relatively restricted area surrounding it. For UAV launch from a charging pad in a cluster, the restricted area could be just a portion of the cluster that includes the charging pad and one or more fiducials. By way of example, a low-altitude initial vertical flight may be in a range of 1-5 m AGL. The UAV may process one or more images into a form of observational data that effectively distills information descriptive of the charging pad and the one or more fiducials, and then transmit the observational data to a server or computer system in a UAV ground-support network for analysis configured to identify specific clusters and pads from the distilled information. By doing so, the server may thus identify the UAV's launch pad and, from the identification, determine, to a threshold accuracy, the UAV's initial geolocation and orientation. Advantageously, the identification and determination may be made using UAV observations obtained prior to location calibration of the UAV.

The server may then use the determined geolocation and orientation of the UAV to devise a mission plan for the UAV, and to provide the UAV with the position/orientation determination, thereby enabling the UAV to calibrate its navigation system in preparation for flight. The UAV may also confirm the accuracy of the determined position/orientation determination upon ascending to a cruising or mission AGL from which it may check its calibrations against a GPS position/orientation determination. The UAV and/or server may take one or more corrective actions if the UAV finds greater than a threshold disagreement between the server-based determination and GPS-based determination.

Further details and advantages of example embodiments are described below.

Herein, the terms “uncrewed aerial vehicle” and “UAV” refer to any autonomous or semi-autonomous vehicle that is capable of performing some functions without a physically present human pilot. As noted above, the term “unmanned” may sometimes be used instead of, or in addition to, “uncrewed,” and it should be understood that both terms have the same meaning, and may be used interchangeably.

A UAV can take various forms. For example, a UAV may take the form of a fixed-wing aircraft, a glider aircraft, a tail-sitter aircraft, a jet aircraft, a ducted fan aircraft, a lighter-than-air dirigible such as a blimp or steerable balloon, a rotorcraft such as a helicopter or multicopter, and/or an ornithopter, among other possibilities. Further, the terms “drone,” “uncrewed aerial vehicle system” (UAVS), or “uncrewed aerial system” (UAS) may also be used to refer to a UAV.

1 FIG.A 100 100 102 104 106 102 102 100 102 108 104 110 112 106 106 114 106 112 114 106 is an isometric view of an example UAV. UAVincludes wing, booms, and a fuselage. Wingsmay be non-flying stationary and may generate lift based on the wing shape and the UAV's forward airspeed. For instance, the two wingsmay have an airfoil-shaped cross section to produce an aerodynamic force on UAV. In some embodiments, wingmay carry horizontal propulsion units, and boomsmay carry vertical propulsion units. In operation, power for the propulsion units may be provided from a battery compartmentof fuselage. In some embodiments, fuselagealso includes an avionics compartment, an additional battery compartment (not shown) and/or a delivery unit (not shown, e.g., a winch system) for handling the payload. In some embodiments, fuselageis modular, and two or more compartments (e.g., battery compartment, avionics compartment, other payload and delivery compartments) are detachable from each other and securable to each other (e.g., mechanically, magnetically, or otherwise) to contiguously form at least a portion of fuselage.

104 116 100 102 117 In some embodiments, boomsterminate in ruddersfor improved yaw control of UAV. Further, wingsmay terminate in wing tipsfor improved control of lift of the UAV.

100 102 104 108 110 In the illustrated configuration, UAVincludes a structural frame. The structural frame may be referred to as a “structural H-frame” or an “H-frame” (not shown) of the UAV. The H-frame may include, within wings, a wing spar (not shown) and, within booms, boom carriers (not shown). In some embodiments the wing spar and the boom carriers may be made of carbon fiber, hard plastic, aluminum, light metal alloys, or other materials. The wing spar and the boom carriers may be connected with clamps. The wing spar may include pre-drilled holes for horizontal propulsion units, and the boom carriers may include pre-drilled holes for vertical propulsion units.

106 106 102 106 100 106 118 106 106 118 106 100 In some embodiments, fuselagemay be removably attached to the H-frame (e.g., attached to the wing spar by clamps, configured with grooves, protrusions or other features to mate with corresponding H-frame features, etc.). In other embodiments, fuselagesimilarly may be removably attached to wings. The removable attachment of fuselagemay improve quality and or modularity of UAV. For example, electrical/mechanical components and/or subsystems of fuselagemay be tested separately from, and before being attached to, the H-frame. Similarly, printed circuit boards (PCBs)may be tested separately from, and before being attached to, the boom carriers, therefore eliminating defective parts/subassemblies prior to completing the UAV. For example, components of fuselage(e.g., avionics, battery unit, delivery units, an additional battery compartment, etc.) may be electrically tested before fuselageis mounted to the H-frame. Furthermore, the motors and the electronics of PCBsmay also be electrically tested before the final assembly. Generally, the identification of the defective parts and subassemblies early in the assembly process lowers the overall cost and lead time of the UAV. Furthermore, different types/models of fuselagemay be attached to the H-frame, therefore improving the modularity of the design. Such modularity allows these various parts of UAVto be upgraded without a substantial overhaul to the manufacturing process.

In some embodiments, a wing shell and boom shells may be attached to the H-frame by adhesive elements (e.g., adhesive tape, double-sided adhesive tape, glue, etc.). Therefore, multiple shells may be attached to the H-frame instead of having a monolithic body sprayed onto the H-frame. In some embodiments, the presence of the multiple shells reduces the stresses induced by the coefficient of thermal expansion of the structural frame of the UAV. As a result, the UAV may have better dimensional accuracy and/or improved reliability.

Moreover, in at least some embodiments, the same H-frame may be used with the wing shell and/or boom shells having different size and/or design, therefore improving the modularity and versatility of the UAV designs. The wing shell and/or the boom shells may be made of relatively light polymers (e.g., closed cell foam) covered by the harder, but relatively thin, plastic skins.

106 118 106 102 104 100 100 119 108 110 100 The power and/or control signals from fuselagemay be routed to PCBsthrough cables running through fuselage, wings, and booms. In the illustrated embodiment, UAVhas four PCBs, but other numbers of PCBs are also possible. For example, UAVmay include two PCBs, one per the boom. The PCBs carry electronic componentsincluding, for example, power converters, controllers, memory, passive components, etc. In operation, propulsion unitsandof UAVare electrically connected to the PCBs.

1 FIG. 102 104 108 110 104 100 100 102 104 Many variations on the illustrated UAV are possible. For instance, fixed-wing UAVs may include more or fewer rotor units (vertical or horizontal), and/or may utilize a ducted fan or multiple ducted fans for propulsion. Further, UAVs with more wings (e.g., an “x-wing” configuration with four wings), are also possible. Althoughillustrates two wings, two booms, two horizontal propulsion units, and six vertical propulsion unitsper boom, it should be appreciated that other variants of UAVmay be implemented with more or less of these components. For example, UAVmay include four wings, four booms, and more or less propulsion units (horizontal or vertical).

1 FIG.B 120 120 122 124 120 126 128 130 132 Similarly,shows another example of a fixed-wing UAV. The fixed-wing UAVincludes a fuselage, two wingswith an airfoil-shaped cross section to provide lift for the UAV, a vertical stabilizer(or fin) to stabilize the plane's yaw (turn left or right), a horizontal stabilizer(also referred to as an elevator or tailplane) to stabilize pitch (tilt up or down), landing gear, and a propulsion unit, which can include a motor, shaft, and propeller.

1 FIG.C 1 1 FIGS.A andB 1 FIG.C 140 142 144 146 148 142 shows an example of a UAVwith a propeller in a pusher configuration. The term “pusher” refers to the fact that a propulsion unitis mounted at the back of the UAV and “pushes” the vehicle forward, in contrast to the propulsion unit being mounted at the front of the UAV. Similar to the description provided for,depicts common structures used in a pusher plane, including a fuselage, two wings, vertical stabilizers, and the propulsion unit, which can include a motor, shaft, and propeller.

1 FIG.D 1 FIG.D 160 160 162 160 162 160 shows an example of a tail-sitter UAV. In the illustrated example, the tail-sitter UAVhas fixed wingsto provide lift and allow the UAVto glide horizontally (e.g., along the x-axis, in a position that is approximately perpendicular to the position shown in). However, the fixed wingsalso allow the tail-sitter UAVto take off and land vertically on its own.

160 164 162 160 160 166 160 168 170 166 160 For example, at a launch site, the tail-sitter UAVmay be positioned vertically (as shown) with its finsand/or wingsresting on the ground and stabilizing the UAVin the vertical position. The tail-sitter UAVmay then take off by operating its propellersto generate an upward thrust (e.g., a thrust that is generally along the y-axis). Once at a suitable altitude, the tail-sitter UAVmay use its flapsto reorient itself in a horizontal position, such that its fuselageis closer to being aligned with the x-axis than the y-axis. Positioned horizontally, the propellersmay provide forward thrust so that the tail-sitter UAVcan fly in a similar manner as a typical airplane.

Many variations on the illustrated fixed-wing UAVs are possible. For instance, fixed-wing UAVs may include more or fewer propellers, and/or may utilize a ducted fan or multiple ducted fans for propulsion. Further, UAVs with more wings (e.g., an “x-wing” configuration with four wings), with fewer wings, or even with no wings, are also possible.

1 FIG.E 180 180 182 180 As noted above, some embodiments may involve other types of UAVs, in addition to or in the alternative to fixed-wing UAVs. For instance,shows an example of a rotorcraft that is commonly referred to as a multicopter. The multicoptermay also be referred to as a quadcopter, as it includes four rotors. It should be understood that example embodiments may involve a rotorcraft with more or fewer rotors than the multicopter. For example, a helicopter typically has two rotors. Other examples with three or more rotors are possible as well. Herein, the term “multicopter” refers to any rotorcraft having more than two rotors, and the term “helicopter” refers to rotorcraft having two rotors.

180 182 180 182 184 182 180 180 Referring to the multicopterin greater detail, the four rotorsprovide propulsion and maneuverability for the multicopter. More specifically, each rotorincludes blades that are attached to a motor. Configured as such, the rotorsmay allow the multicopterto take off and land vertically, to maneuver in any direction, and/or to hover. Further, the pitch of the blades may be adjusted as a group and/or differentially, and may allow the multicopterto control its pitch, roll, yaw, and/or altitude.

It should be understood that references herein to an “uncrewed” aerial vehicle or UAV can apply equally to autonomous and semi-autonomous aerial vehicles. In an autonomous implementation, all functionality of the aerial vehicle is automated; e.g., pre-programmed or controlled via real-time computer functionality that responds to input from various sensors and/or pre-determined information. In a semi-autonomous implementation, some functions of an aerial vehicle may be controlled by a human operator, while other functions are carried out autonomously. Further, in some embodiments, a UAV may be configured to allow a remote operator to take over functions that can otherwise be controlled autonomously by the UAV. Yet further, a given type of function may be controlled remotely at one level of abstraction and performed autonomously at another level of abstraction. For example, a remote operator could control high level navigation decisions for a UAV, such as by specifying that the UAV should travel from one location to another (e.g., from a warehouse in a suburban area to a delivery address in a nearby city), while the UAV's navigation system autonomously controls more fine-grained navigation decisions, such as the specific route to take between the two locations, specific flight controls to achieve the route and avoid obstacles while navigating the route, and so on.

More generally, it should be understood that the example UAVs described herein are not intended to be limiting. Example embodiments may relate to, be implemented within, or take the form of any type of uncrewed aerial vehicle.

2 FIG. 1 1 FIGS.A-E 200 200 100 120 140 160 180 200 is a simplified block diagram illustrating components of a UAV, according to an example embodiment. UAVmay take the form of, or be similar in form to, one of the UAVs,,,, anddescribed in reference to. However, UAVmay also take other forms.

200 200 202 204 206 UAVmay include various types of sensors, and may include a computing system configured to provide the functionality described herein. In the illustrated embodiment, the sensors of UAVinclude an inertial measurement unit (IMU), ultrasonic sensor(s), and a GPS, among other possible sensors and sensing systems.

200 208 208 208 212 210 In the illustrated embodiment, UAValso includes one or more processors. A processormay be a general-purpose processor or a special purpose processor (e.g., digital signal processors, application specific integrated circuits, etc.). The one or more processorscan be configured to execute computer-readable program instructionsthat are stored in the data storageand are executable to provide the functionality of a UAV described herein.

210 208 208 210 210 The data storagemay include or take the form of one or more computer-readable storage media that can be read or accessed by at least one processor. The one or more computer-readable storage media can include volatile and/or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which can be integrated in whole or in part with at least one of the one or more processors. In some embodiments, the data storagecan be implemented using a single physical device (e.g., one optical, magnetic, organic or other memory or disc storage unit), while in other embodiments, the data storagecan be implemented using two or more physical devices.

210 212 200 210 212 212 214 216 As noted, the data storagecan include computer-readable program instructionsand perhaps additional data, such as diagnostic data of the UAV. As such, the data storagemay include program instructionsto perform or facilitate some or all of the UAV functionality described herein. For instance, in the illustrated embodiment, program instructionsinclude a navigation moduleand a tether control module.

A. Sensors

202 200 202 In an illustrative embodiment, IMUmay include both an accelerometer and a gyroscope, which may be used together to determine an orientation of the UAV. In particular, the accelerometer can measure the orientation of the vehicle with respect to earth, while the gyroscope measures the rate of rotation around an axis. IMUs are commercially available in low-cost, low-power packages. For instance, an IMUmay take the form of or include a miniaturized MicroElectroMechanical System (MEMS) or a NanoElectroMechanical System (NEMS). Other types of IMUs may also be utilized.

202 200 An IMUmay include other sensors, in addition to accelerometers and gyroscopes, which may help to better determine position and/or help to increase autonomy of the UAV. Two examples of such sensors are magnetometers and pressure sensors. In some embodiments, a UAV may include a low-power, digital 3-axis magnetometer, which can be used to realize an orientation independent electronic compass for accurate heading information. However, other types of magnetometers may be utilized as well. Other examples are also possible. Further, note that a UAV could include some or all of the above-described inertia sensors as separate components from an IMU.

200 200 UAVmay also include a pressure sensor or barometer, which can be used to determine the altitude of the UAV. Alternatively, other sensors, such as sonic altimeters or radar altimeters, can be used to provide an indication of altitude, which may help to improve the accuracy of and/or prevent drift of an IMU.

200 200 204 204 In a further aspect, UAVmay include one or more sensors that allow the UAV to sense objects in the environment. For instance, in the illustrated embodiment, UAVincludes ultrasonic sensor(s). Ultrasonic sensor(s)can determine the distance to an object by generating sound waves and determining the time interval between transmission of the wave and receiving the corresponding echo off an object. A typical application of an ultrasonic sensor for uncrewed vehicles or IMUs is low-level altitude control and obstacle avoidance. An ultrasonic sensor can also be used for vehicles that need to hover at a certain height or need to be capable of detecting obstacles. Other systems can be used to determine, sense the presence of, and/or determine the distance to nearby objects, such as a light detection and ranging (LIDAR) system, laser detection and ranging (LADAR) system, and/or an infrared or forward-looking infrared (FLIR) system, among other possibilities.

200 200 In some embodiments, UAVmay also include one or more imaging system(s). For example, one or more still and/or video cameras may be utilized by UAVto capture image data from the UAV's environment. As a specific example, charge-coupled device (CCD) cameras or complementary metal-oxide-semiconductor (CMOS) cameras can be used with uncrewed vehicles. Such imaging sensor(s) have numerous possible applications, such as obstacle avoidance, localization techniques, ground tracking for more accurate navigation (e.g., by applying optical flow techniques to images), video feedback, and/or image recognition and processing, among other possibilities.

200 206 206 200 200 206 UAVmay also include a GPS receiver. The GPS receivermay be configured to provide data that is typical of well-known GPS systems, such as the GPS coordinates of the UAV. Such GPS data may be utilized by the UAVfor various functions. As such, the UAV may use its GPS receiverto help navigate to the caller's location, as indicated, at least in part, by the GPS coordinates provided by their mobile device. Other examples are also possible.

B. Navigation and Location Determination

214 200 214 The navigation modulemay provide functionality that allows the UAVto, e.g., move about its environment and reach a desired location. To do so, the navigation modulemay control the altitude and/or direction of flight by controlling the mechanical features of the UAV that affect flight (e.g., its rudder(s), elevator(s), aileron(s), and/or the speed of its propeller(s)).

200 214 200 200 200 200 200 In order to navigate the UAVto a target location, the navigation modulemay implement various navigation techniques, such as map-based navigation and localization-based navigation, for instance. With map-based navigation, the UAVmay be provided with a map of its environment, which may then be used to navigate to a particular location on the map. With localization-based navigation, the UAVmay be capable of navigating in an unknown environment using localization. Localization-based navigation may involve the UAVbuilding its own map of its environment and calculating its position within the map and/or the position of objects in the environment. For example, as a UAVmoves throughout its environment, the UAVmay continuously use localization to update its map of the environment. This continuous mapping process may be referred to as simultaneous localization and mapping (SLAM). Other navigation techniques may also be utilized.

214 214 200 In some embodiments, the navigation modulemay navigate using a technique that relies on waypoints. In particular, waypoints are sets of coordinates that identify points in physical space. For instance, an air-navigation waypoint may be defined by a certain latitude, longitude, and altitude. Accordingly, navigation modulemay cause UAVto move from waypoint to waypoint, in order to ultimately travel to a final destination (e.g., a final waypoint in a sequence of waypoints).

214 200 228 In a further aspect, the navigation moduleand/or other components and systems of the UAVmay be configured for “localization” to more precisely navigate to the scene of a target location. More specifically, it may be desirable in certain situations for a UAV to be within a threshold distance of the target location where a payloadis being delivered by a UAV (e.g., within a few feet of the target destination). To this end, a UAV may use a two-tiered approach in which it uses a more-general location-determination technique to navigate to a general area that is associated with the target location, and then use a more-refined location-determination technique to identify and/or navigate to the target location within the general area.

200 228 200 200 200 For example, the UAVmay navigate to the general area of a target destination where a payloadis being delivered using waypoints and/or map-based navigation. The UAV may then switch to a mode in which it utilizes a localization process to locate and travel to a more specific location. For instance, if the UAVis to deliver a payload to a user's home, the UAVmay need to be substantially close to the target location in order to avoid delivery of the payload to undesired areas (e.g., onto a roof, into a pool, onto a neighbor's property, etc.). However, a GPS signal may only get the UAVso far (e.g., within a block of the user's home). A more precise location-determination technique may then be used to find the specific target location.

200 200 204 214 Various types of location-determination techniques may be used to accomplish localization of the target delivery location once the UAVhas navigated to the general area of the target delivery location. For instance, the UAVmay be equipped with one or more sensory systems, such as, for example, ultrasonic sensors, infrared sensors (not shown), and/or other sensors, which may provide input that the navigation moduleutilizes to navigate autonomously or semi-autonomously to the specific target location.

200 200 200 200 200 As another example, once the UAVreaches the general area of the target delivery location (or of a moving subject such as a person or their mobile device), the UAVmay switch to a “fly-by-wire” mode where it is controlled, at least in part, by a remote operator, who can navigate the UAVto the specific target location. To this end, sensory data from the UAVmay be sent to the remote operator to assist them in navigating the UAVto the specific location.

200 200 200 200 As yet another example, the UAVmay include a module that is able to signal to a passer-by for assistance in either reaching the specific target delivery location; for example, the UAVmay display a visual message requesting such assistance in a graphic display, play an audio message or tone through speakers to indicate the need for such assistance, among other possibilities. Such a visual or audio message might indicate that assistance is needed in delivering the UAVto a particular person or a particular location, and might provide information to assist the passer-by in delivering the UAVto the person or location (e.g., a description or picture of the person or location, and/or the person or location's name), among other possibilities. Such a feature can be useful in a scenario in which the UAV is unable to use sensory functions or another location-determination technique to reach the specific target location. However, this feature is not limited to such scenarios.

200 200 200 200 200 200 In some embodiments, once the UAVarrives at the general area of a target delivery location, the UAVmay utilize a beacon from a user's remote device (e.g., the user's mobile phone) to locate the person. Such a beacon may take various forms. As an example, consider the scenario where a remote device, such as the mobile phone of a person who requested a UAV delivery, is able to send out directional signals (e.g., via an RF signal, a light signal and/or an audio signal). In this scenario, the UAVmay be configured to navigate by “sourcing” such directional signals—in other words, by determining where the signal is strongest and navigating accordingly. As another example, a mobile device can emit a frequency, either in the human range or outside the human range, and the UAVcan listen for that frequency and navigate accordingly. As a related example, if the UAVis listening for spoken commands, then the UAVcould utilize spoken statements, such as “I'm over here!” to source the specific location of the person requesting delivery of a payload.

200 200 200 200 200 200 200 200 In an alternative arrangement, a navigation module may be implemented at a remote computing device, which communicates wirelessly with the UAV. The remote computing device may receive data indicating the operational state of the UAV, sensor data from the UAVthat allows it to assess the environmental conditions being experienced by the UAV, and/or location information for the UAV. Provided with such information, the remote computing device may determine latitudinal and/or directional adjustments that should be made by the UAVand/or may determine how the UAVshould adjust its mechanical features (e.g., its rudder(s), elevator(s), aileron(s), and/or the speed of its propeller(s)) in order to effectuate such movements. The remote computing system may then communicate such adjustments to the UAVso it can move in the determined manner.

C. Communication Systems

200 218 218 200 In a further aspect, the UAVincludes one or more communication systems. The communications systemsmay include one or more wireless interfaces and/or one or more wireline interfaces, which allow the UAVto communicate via one or more networks. Such wireless interfaces may provide for communication under one or more wireless communication protocols, such as Bluetooth, WiFi (e.g., an IEEE 802.11 protocol), Long-Term Evolution (LTE), WiMAX (e.g., an IEEE 802.16 standard), a radio-frequency ID (RFID) protocol, near-field communication (NFC), and/or other wireless communication protocols. Such wireline interfaces may include an Ethernet interface, a Universal Serial Bus (USB) interface, or similar interface to communicate via a wire, a twisted pair of wires, a coaxial cable, an optical link, a fiber-optic link, or other physical connection to a wireline network.

200 218 200 200 200 In some embodiments, a UAVmay include communication systemsthat allow for both short-range communication and long-range communication. For example, the UAVmay be configured for short-range communications using Bluetooth and for long-range communications under a CDMA protocol. In such an embodiment, the UAVmay be configured to function as a “hot spot;” or in other words, as a gateway or proxy between a remote support device and one or more data networks, such as a cellular network and/or the Internet. Configured as such, the UAVmay facilitate data communications that the remote support device would otherwise be unable to perform by itself.

200 200 For example, the UAVmay provide a WiFi connection to a remote device, and serve as a proxy or gateway to a cellular service provider's data network, which the UAV might connect to under an LTE or a 3G protocol, for instance. The UAVcould also serve as a proxy or gateway to a high-altitude balloon network, a satellite network, or a combination of these networks, among others, which a remote device might not be able to otherwise access.

D. Power Systems

200 220 220 200 In a further aspect, the UAVmay include power system(s). The power systemmay include one or more batteries for providing power to the UAV. In one example, the one or more batteries may be rechargeable and each battery may be recharged via a wired connection between the battery and a power supply and/or via a wireless charging system, such as an inductive charging system that applies an external time-varying magnetic field to an internal battery.

E. Payload Delivery

200 228 228 200 200 228 The UAVmay employ various systems and configurations in order to transport and deliver a payload. In some implementations, the payloadof a given UAVmay include or take the form of a “package” designed to transport various goods to a target delivery location. For example, the UAVcan include a compartment, in which an item or items may be transported. Such a package may include one or more food items, purchased goods, medical items, or any other object(s) having a size and weight suitable to be transported between two locations by the UAV. In other embodiments, a payloadmay simply be the one or more items that are being delivered (e.g., without any package housing the items).

228 In some embodiments, the payloadmay be attached to the UAV and located substantially outside of the UAV during some or all of a flight by the UAV. For example, the package may be tethered or otherwise releasably attached below the UAV during flight to a target location. In some embodiments, the package may include various features that protect its contents from the environment, reduce aerodynamic drag on the system, and prevent the contents of the package from shifting during UAV flight. In other embodiments, the package may be a standard shipping package that is not specifically tailored for UAV flight.

221 216 228 221 224 224 228 226 224 222 222 216 222 224 226 224 228 222 2 FIG. In order to deliver the payload, the UAV may include a winch systemcontrolled by the tether control modulein order to lower the payloadto the ground while the UAV hovers above. As shown in, the winch systemmay include a tether, and the tethermay be coupled to the payloadby a payload retriever. The tethermay be wound on a spool that is coupled to a motorof the UAV. The motormay take the form of a DC motor (e.g., a servo motor) that can be actively controlled by a speed controller. The tether control modulecan control the speed controller to cause the motorto rotate the spool, thereby unwinding or retracting the tetherand lowering or raising the payload retriever. In practice, the speed controller may output a desired operating rate (e.g., a desired RPM) for the spool, which may correspond to the speed at which the tetherand payloadshould be lowered towards the ground. The motormay then rotate the spool so that it maintains the desired operating rate.

222 216 222 216 In order to control the motorvia the speed controller, the tether control modulemay receive data from a speed sensor (e.g., an encoder) configured to convert a mechanical position to a representative analog or digital signal. In particular, the speed sensor may include a rotary encoder that may provide information related to rotary position (and/or rotary movement) of a shaft of the motor or the spool coupled to the motor, among other possibilities. Moreover, the speed sensor may take the form of an absolute encoder and/or an incremental encoder, among others. So in an example implementation, as the motorcauses rotation of the spool, a rotary encoder may be used to measure this rotation. In doing so, the rotary encoder may be used to convert a rotary position to an analog or digital electronic signal used by the tether control moduleto determine the amount of rotation of the spool from a fixed reference angle and/or to an analog or digital electronic signal that is representative of a new rotary position, among other options. Other examples are also possible.

216 222 222 222 222 222 Based on the data from the speed sensor, the tether control modulemay determine a rotational speed of the motorand/or the spool and responsively control the motor(e.g., by increasing or decreasing an electrical current supplied to the motor) to cause the rotational speed of the motorto match a desired speed. When adjusting the motor current, the magnitude of the current adjustment may be based on a proportional-integral-derivative (PID) calculation using the determined and desired speeds of the motor. For instance, the magnitude of the current adjustment may be based on a present difference, a past difference (based on accumulated error over time), and a future difference (based on current rates of change) between the determined and desired speeds of the spool.

216 224 228 228 216 224 200 224 200 224 222 224 222 224 In some embodiments, the tether control modulemay vary the rate at which the tetherand payloadare lowered to the ground. For example, the speed controller may change the desired operating rate according to a variable deployment-rate profile and/or in response to other factors in order to change the rate at which the payloaddescends toward the ground. To do so, the tether control modulemay adjust an amount of braking or an amount of friction that is applied to the tether. For example, to vary the tether deployment rate, the UAVmay include friction pads that can apply a variable amount of pressure to the tether. As another example, the UAVcan include a motorized braking system that varies the rate at which the spool lets out the tether. Such a braking system may take the form of an electromechanical system in which the motoroperates to slow the rate at which the spool lets out the tether. Further, the motormay vary the amount by which it adjusts the speed (e.g., the RPM) of the spool, and thus may vary the deployment rate of the tether. Other examples are also possible.

216 222 222 222 224 200 224 222 224 200 In some embodiments, the tether control modulemay be configured to limit the motor current supplied to the motorto a maximum value. With such a limit placed on the motor current, there may be situations where the motorcannot operate at the desired operation specified by the speed controller. For instance, as discussed in more detail below, there may be situations where the speed controller specifies a desired operating rate at which the motorshould retract the tethertoward the UAV, but the motor current may be limited such that a large enough downward force on the tetherwould counteract the retracting force of the motorand cause the tetherto unwind instead. And as further discussed below, a limit on the motor current may be imposed and/or altered depending on an operational state of the UAV.

216 224 228 222 224 228 224 224 200 216 222 224 228 216 222 216 222 216 220 222 228 224 224 226 200 224 In some embodiments, the tether control modulemay be configured to determine a status of the tetherand/or the payloadbased on the amount of current supplied to the motor. For instance, if a downward force is applied to the tether(e.g., if the payloadis attached to the tetheror if the tethergets snagged on an object when retracting toward the UAV), the tether control modulemay need to increase the motor current in order to cause the determined rotational speed of the motorand/or spool to match the desired speed. Similarly, when the downward force is removed from the tether(e.g., upon delivery of the payloador removal of a tether snag), the tether control modulemay need to decrease the motor current in order to cause the determined rotational speed of the motorand/or spool to match the desired speed. As such, the tether control modulemay be configured to monitor the current supplied to the motor. For instance, the tether control modulecould determine the motor current based on sensor data received from a current sensor of the motor or a current sensor of the power system. In any case, based on the current supplied to the motor, determine if the payloadis attached to the tether, if someone or something is pulling on the tether, and/or if the payload retrieveris pressing against the UAVafter retracting the tether. Other examples are possible as well.

228 226 228 224 228 226 224 222 During delivery of the payload, the payload retrievercan be configured to secure the payloadwhile being lowered from the UAV by the tether, and can be further configured to release the payloadupon reaching ground level. The payload retrievercan then be retracted to the UAV by reeling in the tetherusing the motor.

228 228 228 228 228 228 228 In some implementations, the payloadmay be passively released once it is lowered to the ground. For example, a passive release mechanism may include one or more swing arms adapted to retract into and extend from a housing. An extended swing arm may form a hook on which the payloadmay be attached. Upon lowering the release mechanism and the payloadto the ground via a tether, a gravitational force as well as a downward inertial force on the release mechanism may cause the payloadto detach from the hook allowing the release mechanism to be raised upwards toward the UAV. The release mechanism may further include a spring mechanism that biases the swing arm to retract into the housing when there are no other external forces on the swing arm. For instance, a spring may exert a force on the swing arm that pushes or pulls the swing arm toward the housing such that the swing arm retracts into the housing once the weight of the payloadno longer forces the swing arm to extend from the housing. Retracting the swing arm into the housing may reduce the likelihood of the release mechanism snagging the payloador other nearby objects when raising the release mechanism toward the UAV upon delivery of the payload.

Active payload release mechanisms are also possible. For example, sensors such as a barometric pressure based altimeter and/or accelerometers may help to detect the position of the release mechanism (and the payload) relative to the ground. Data from the sensors can be communicated back to the UAV and/or a control system over a wireless link and used to help in determining when the release mechanism has reached ground level (e.g., by detecting a measurement with the accelerometer that is characteristic of ground impact). In other examples, the UAV may determine that the payload has reached the ground based on a weight sensor detecting a threshold low downward force on the tether and/or based on a threshold low measurement of power drawn by the winch when lowering the payload.

200 200 Other systems and techniques for delivering a payload, in addition or in the alternative to a tethered delivery system are also possible. For example, a UAVcould include an air-bag drop system or a parachute drop system. Alternatively, a UAVcarrying a payload could simply land on the ground at a delivery location. Other examples are also possible.

3 FIG. 300 UAV systems may be implemented in order to provide various UAV-related services. In particular, UAVs may be provided at a number of different launch sites that may be in communication with regional and/or central control systems. Such a distributed UAV system may allow UAVs to be quickly deployed to provide services across a large geographic area (e.g., that is much larger than the flight range of any single UAV). For example, UAVs capable of carrying payloads may be distributed at a number of launch sites across a large geographic area (possibly even throughout an entire country, or even worldwide), in order to provide on-demand transport of various items to locations throughout the geographic area.is a simplified block diagram illustrating a distributed UAV system, according to an example embodiment.

300 302 304 302 304 304 In the illustrative UAV system, an access systemmay allow for interaction with, control of, and/or utilization of a network of UAVs. In some embodiments, an access systemmay be a computing system that allows for human-controlled dispatch of UAVs. As such, the control system may include or otherwise provide a user interface through which a user can access and/or control the UAVs.

304 302 304 In some embodiments, dispatch of the UAVsmay additionally or alternatively be accomplished via one or more automated processes. For instance, the access systemmay dispatch one of the UAVsto transport a payload to a target location, and the UAV may autonomously navigate to the target location by utilizing various on-board sensors, such as a GPS receiver and/or other various navigational sensors.

302 302 302 304 304 302 304 Further, the access systemmay provide for remote operation of a UAV. For instance, the access systemmay allow an operator to control the flight of a UAV via its user interface. As a specific example, an operator may use the access systemto dispatch a UAVto a target location. The UAVmay then autonomously navigate to the general area of the target location. At this point, the operator may use the access systemto take control of the UAVand navigate the UAV to the target location (e.g., to a particular person to whom a payload is being transported). Other examples of remote operation of a UAV are also possible.

304 304 300 304 304 304 1 1 FIGS.A-E In an illustrative embodiment, the UAVsmay take various forms. For example, each of the UAVsmay be a UAV such as those illustrated in. However, UAV systemmay also utilize other types of UAVs without departing from the scope of the invention. In some implementations, all of the UAVsmay be of the same or a similar configuration. However, in other implementations, the UAVsmay include a number of different types of UAVs. For instance, the UAVsmay include a number of types of UAVs, with each type of UAV being configured for a different type or types of payload delivery capabilities.

300 306 306 306 306 306 The UAV systemmay further include a remote device, which may take various forms. Generally, the remote devicemay be any device through which a direct or indirect request to dispatch a UAV can be made. (Note that an indirect request may involve any communication that may be responded to by dispatching a UAV, such as requesting a package delivery). In an example embodiment, the remote devicemay be a mobile phone, tablet computer, laptop computer, personal computer, or any network-connected computing device. Further, in some instances, the remote devicemay not be a computing device. As an example, a standard telephone, which allows for communication via plain old telephone service (POTS), may serve as the remote device. Other types of remote devices are also possible.

306 302 308 306 302 302 Further, the remote devicemay be configured to communicate with access systemvia one or more types of communication network(s). For example, the remote devicemay communicate with the access system(or a human operator of the access system) by communicating over a POTS network, a cellular network, and/or a data network such as the Internet. Other types of networks may also be utilized.

306 300 In some embodiments, the remote devicemay be configured to allow a user to request delivery of one or more items to a desired location. For example, a user could request UAV delivery of a package to their home via their mobile phone, tablet, or laptop. As another example, a user could request dynamic delivery to wherever they are located at the time of delivery. To provide such dynamic delivery, the UAV systemmay receive location information (e.g., GPS coordinates, etc.) from the user's mobile phone, or any other device on the user's person, such that a UAV can navigate to the user's location (as indicated by their mobile phone).

310 302 310 310 312 310 302 In an illustrative arrangement, the central dispatch systemmay be a server or group of servers, which is configured to receive dispatch messages requests and/or dispatch instructions from the access system. Such dispatch messages may request or instruct the central dispatch systemto coordinate the deployment of UAVs to various target locations. The central dispatch systemmay be further configured to route such requests or instructions to one or more local dispatch systems. To provide such functionality, the central dispatch systemmay communicate with the access systemvia a data network, such as the Internet or a private network that is established for communications between access systems and automated dispatch systems.

310 304 312 310 304 312 304 304 312 304 In the illustrated configuration, the central dispatch systemmay be configured to coordinate the dispatch of UAVsfrom a number of different local dispatch systems. As such, the central dispatch systemmay keep track of which UAVsare located at which local dispatch systems, which UAVsare currently available for deployment, and/or which services or operations each of the UAVsis configured for (in the event that a UAV fleet includes multiple types of UAVs configured for different services and/or operations). Additionally or alternatively, each local dispatch systemmay be configured to track which of its associated UAVsare currently available for deployment and/or are currently in the midst of item transport.

310 302 310 304 310 312 312 314 310 312 304 312 In some cases, when the central dispatch systemreceives a request for UAV-related service (e.g., transport of an item) from the access system, the central dispatch systemmay select a specific UAVto dispatch. The central dispatch systemmay accordingly instruct the local dispatch systemthat is associated with the selected UAV to dispatch the selected UAV. The local dispatch systemmay then operate its associated deployment systemto launch the selected UAV. In other cases, the central dispatch systemmay forward a request for a UAV-related service to a local dispatch systemthat is near the location where the support is requested and leave the selection of a particular UAVto the local dispatch system.

312 314 312 314 304 312 312 314 304 In an example configuration, the local dispatch systemmay be implemented as a computing system at the same location as the deployment system(s)that it controls. For example, the local dispatch systemmay be implemented by a computing system installed at a building, such as a warehouse, where the deployment system(s)and UAV(s)that are associated with the particular local dispatch systemare also located. In other embodiments, the local dispatch systemmay be implemented at a location that is remote to its associated deployment system(s)and UAV(s).

300 306 310 306 300 310 312 Numerous variations on and alternatives to the illustrated configuration of the UAV systemare possible. For example, in some embodiments, a user of the remote devicecould request delivery of a package directly from the central dispatch system. To do so, an application may be implemented on the remote devicethat allows the user to provide information regarding a requested delivery, and generate and send a data message to request that the UAV systemprovide the delivery. In such an embodiment, the central dispatch systemmay include automated functionality to handle requests that are generated by such an application, evaluate such requests, and, if appropriate, coordinate with an appropriate local dispatch systemto deploy a UAV.

310 312 302 314 310 312 302 314 Further, some or all of the functionality that is attributed herein to the central dispatch system, the local dispatch system(s), the access system, and/or the deployment system(s)may be combined in a single system, implemented in a more complex system, and/or redistributed among the central dispatch system, the local dispatch system(s), the access system, and/or the deployment system(s)in various ways.

312 314 312 314 310 312 310 312 Yet further, while each local dispatch systemis shown as having two associated deployment systems, a given local dispatch systemmay alternatively have more or fewer associated deployment systems. Similarly, while the central dispatch systemis shown as being in communication with two local dispatch systems, the central dispatch systemmay alternatively be in communication with more or fewer local dispatch systems.

314 314 304 314 304 304 In a further aspect, the deployment systemsmay take various forms. In general, the deployment systemsmay take the form of or include systems for physically launching one or more of the UAVs. Such launch systems may include features that provide for an automated UAV launch and/or features that allow for a human-assisted UAV launch. Further, the deployment systemsmay each be configured to launch one particular UAV, or to launch multiple UAVs.

314 The deployment systemsmay further be configured to provide additional functions, including for example, diagnostic-related functions such as verifying system functionality of the UAV, verifying functionality of devices that are housed within a UAV (e.g., a payload delivery apparatus), and/or maintaining devices or other items that are housed in the UAV (e.g., by monitoring a status of a payload such as its temperature, weight, etc.).

314 304 312 314 314 314 312 In some embodiments, the deployment systemsand their corresponding UAVs(and possibly associated local dispatch systems) may be strategically distributed throughout an area such as a city. For example, the deployment systemsmay be strategically distributed such that each deployment systemis proximate to one or more payload pickup locations (e.g., near a restaurant, store, or warehouse). However, the deployment systems(and possibly the local dispatch systems) may be distributed in other ways, depending upon the particular implementation. As an additional example, kiosks that allow users to transport packages via UAVs may be installed in various locations. Such kiosks may include UAV launch systems, and may allow a user to provide their package for loading onto a UAV and pay for UAV shipping services, among other possibilities. Other examples are also possible.

300 316 316 316 In a further aspect, the UAV systemmay include or have access to a user-account database. The user-account databasemay include data for a number of user accounts, and which are each associated with one or more persons. For a given user account, the user-account databasemay include data related to or useful in providing UAV-related services. Typically, the user data associated with each user account is optionally provided by an associated user and/or is collected with the associated user's permission.

300 304 300 316 Further, in some embodiments, a person may be required to register for a user account with the UAV system, if they wish to be provided with UAV-related services by the UAVsfrom UAV system. As such, the user-account databasemay include authorization information for a given user account (e.g., a username and password), and/or other information that may be used to authorize access to a user account.

300 302 In some embodiments, a person may associate one or more of their devices with their user account, such that they can access the services of UAV system. For example, when a person uses an associated mobile phone, e.g., to place a call to an operator of the access systemor send a message requesting a UAV-related service to a dispatch system, the phone may be identified via a unique device identification number, and the call or message may then be attributed to the associated user account. Other examples are also possible.

Context-based navigation of UAVs using patterns or constellations of fiducial markers may be described by way of example operation by considering a UAV flying to a designated target charging pad among a cluster of charging pads. In an example usage scenario, such a UAV may have a mission that includes (among possibly other actions) flying to and landing on the designated target charging pad. The example operation, described below, is not intended to be limiting with respect to the scope of example embodiments described herein.

300 314 314 314 314 3 FIG. 3 FIG. The UAV systemillustrated inis an example of a support infrastructure described briefly above in connection with battery charging pads and mission and operations support. In connection with these and other aspects of UAV fleet operation, the deployment system(s)ofmay be considered a jet bridge. More particularly, the deployment system(s)may function as launching and landing sites or platforms for UAVs, and, as shown, may also be communicatively connected with other components of the infrastructure, including those which may support mission operations and data processing/analysis. As such, a deployment systemmay provide a variety of support operations for UAVs, including communications with network-based servers and other infrastructure elements. Deployment system(s)may also include clusters of charging pads.

3 FIG. Although not necessarily shown as distinct infrastructure entities in, charging pad clusters may also be deployed as standalone configurations. In accordance with example embodiments, a standalone charging pad cluster may include a layout of charging pads connected to AC power, and possibly configured to function as little more than landing pads or platforms with battery charging facilities. In practice, charging pad clusters may be located at delivery pick-up sites, such as vendors' warehouses or retail stores, for example. UAVs may be controlled and/or preprogrammed to visit such sites to retrieve items for delivery, and to charge their batteries during their stopovers, but may not necessarily need any other services during each such stop.

4 FIG.A 4 FIG.A 4 FIG.D 400 402 400 illustrates an example charging pad cluster, in accordance with example embodiments. In the illustration, charging padsare represented by squares at regular positions in a square grid (or array) having five charging pads on each side for a total of 25 charging pads. As mentioned above and discussed in more detail below, a charging pad cluster may also include a plurality of fiducial markers at positions distributed across the layout (e.g., grid) of charging pads. For the moment, fiducial markers are omitted from the depiction of charging pad clusterin. However, a distribution of fiducial markers will be depicted in a subsequent figure (see) and discussed in connection therewith.

4 FIG.A 402 402 402 402 x y x y x x Continuing with, axes labeled “X” and “Y” at the lower left of the grid represent a coordinate system for the grid. By way of example, all the charging pads are oriented with their sides parallel to the axes. For purposes of the discussion here, two of the charging pads are distinctly labeled-A and-B. Also by way of example, all the charging padsare identical in size, having dimensions of Dand D, as shown. As also shown, the center-to-center separation of the charging pads is Sin the X-direction and Sin the Y-direction. And again by way of example, the X and Y separations are equal (S=S). In the discussion of example operation below, charging pad-A serves as an example designated target charging pad for a UAV, so it is depicted with a thick black line as a visual cue of its special role in the discussion.

400 400 x y x y 4 FIG.A It should be understood that other configurations of charging pads in a cluster are possible as well. For example, the layout need not necessarily be square or even regular, and the orientation of the charging pads need not necessarily be identical or aligned with the sides of a grid. Further, a cluster need not necessarily include only identically-sized charging pads. For the purposes of the present discussion, however, there is no loss in generality by considering a cluster of identically-sized, square (or rectangular) charging pads configured at regular positions in a square (or rectangular) grid, and all oriented parallel to the sides of the grid. Non-limiting examples of dimensions of an example embodiment of charging pad clusterinclude D=D=1.2 meters (4 feet), and S=S=3 meters (9.8 feet). Note that the charging pad clusterinis not necessarily to scale.

For purposes of the discussion herein, the vicinity of a charging pad cluster will be referred to as its “locale.” Non-limiting examples of a locale of a charging cluster may be a warehouse in which, or adjacent to which, the charging pad cluster is located. Other terms similar to “locale” may also be used to convey its meaning as applied herein, such as “locality,” “region,” and “site,” among others.

402 400 In an example operational scenario such as the one described above, a UAV may intend or be instructed (e.g., according to a mission flight plan) to fly to and land on a designated target charging pad—for example, charging pad-A of charging cluster. The specification of a designated target charging pad from among a cluster may be related to particular operational protocols of a fleet of UAVs. For example, coordination of multiple flight plans of multiple UAVs of a fleet may call for or require advanced assignments of designated charging pads for landing. There could be other or additional reasons as well.

In an example embodiment, the UAV may be configured for both flying predominantly horizontally between different geolocations, and ascending, descending, and hovering vertically over any particular location on the ground. For purposes of the discussion herein, it may be assumed that in flying from ground-point “A” to ground-point “B,” a UAV may ascend vertically from ground-point A, fly predominantly horizontally to a point vertically above ground-point B, then descend to land at ground-point B. Further, the UAV may need to confirm that it has correctly identified ground-point B before descending to land there, and the UAV may possibly adjust its horizontal position if necessary first. However, it should be understood that while the vertical ascent, descent, and hovering operations may be useful and/or helpful in describing and illustrating example operation, they are not necessarily required aspects of example embodiments. In particular, the example operation described below may be adapted to accommodate UAVs that take off and land using runways, similarly to commercial airplanes, for example.

200 400 2 FIG. The UAV, such as UAV, may have a navigation system, as described above, and may use the navigation system when flying to the locale of charging cluster. Although not necessarily shown inor explicitly described in the associated discussion, the navigation system may be configured to operate in more than one mode. In particular, and in accordance with example embodiments, the navigation system may be configured to operate in at least a “coarse-navigation” mode and a “context-based fine-navigation mode,” or just “context-based” navigation mode.

400 In accordance with example embodiments, the coarse-navigation mode may enable the UAV to fly to a location that is within a tolerance of a desired or designated location, so as to arrive at a “coarse-navigation” position or location. The coarse-navigation position may be vertically above an approximate ground position of the designated target charging pad, but insufficiently accurate for the UAV to distinguish its designated target charging pad from one or more other nearby (e.g., neighboring) charging pads. The UAV may then use (e.g., switch to) the context-based navigation mode, which enables the UAV to distinguish its designated target charging pad from the other nearby charging pads based on a comparison of observed constellations of fiducial markers relative to the charging pads with known reference constellations in a reference map of the charging pad cluster. The UAV may then descend and land on its designated target charging pad once it confirms it has correctly identified its designated pad.

402 In an example, the coarse-navigation mode may use a GPS system. Such a system may be subject to limitations of accuracy due to inherent operational factors and/or environmental factors that may degrade or diminish a received GPS signal, for example. Inherent limitations on accuracy could correspond to best-case accuracy or accuracy under best practical environmental and/or operational conditions. Environmental factors could be dependent on the locale of the charging cluster or particular locations within the locale, for example. In practice, environmental factors could affect the likelihood that a given degree of inherent accuracy will be achieved on any given flight. However, the likelihood could also include random factors as well. In any case, in using the coarse-navigation mode to fly to the locale, the UAV may thus arrive to within the tolerance of charging pad-A with a given likelihood. This may be expressed as the UAV having a “Q” percent chance of arriving to within “R” meters of a position directly above the designated target charging pad.

4 FIG.B 4 FIG.B 402 400 402 402 402 is a conceptual illustration of a coarse-navigation position with respect to charging pad-A of charging pad cluster, in accordance with example embodiments. In this illustration, a few different tolerances (“R” in the expression above) of distance from a center point of charging pad-A as viewed from above are represented as concentric rings of grayscale, and the associated likelihoods (“Q” in the expression above) are indicated as percentages within the tolerances. Taking an example in which R is normally distributed, Q=99% could be interpreted as a 3σ (“3-sigma”) statistic. An example 30 tolerance is indicated by a dashed-dotted circle labeled “Coarse-Navigation Accuracy.” For illustrative purposes, and by way of example, the 3σ tolerance is taken to be approximately 3 meters. That is, a UAV using its coarse-navigation mode to fly to charging pad-A may be expected to arrive within 3 meters of a position above charging pad-A with a 99% likelihood. In this example illustration, it also appears that the UAV may be expected to arrive within approximately 1.5, 1.0, and 0.5 meters with approximately 93%, 84%, and 70% likelihoods, respectively. These likelihoods simply express integrals of a normal distribution having σ=1 meter. Note that the concentric rings of grayscale inare not necessarily to scale.

4 FIG.B It should be understood that the example tolerances and likelihoods above are largely hypothetical, and presented for illustrative purposes. Similarly, the assumption of a normal distribution of coarse-navigation arrival locations is also presented by way of example. In some implementations, the actual values and distributions could be different, though the example values might not be unreasonable estimates of actual values that may be found in practice. Putting aside the hypothetical nature of the tolerances and likelihoods above,serves to illustrate that there is a degree of approximation of the coarse-navigation position and a resulting potential for mis-identification of a designated target charging pad.

4 FIG.B 402 402 402 402 Operationally, when a UAV arrives at, or as the UAV is approaching, a charging pad cluster, the UAV may initially identify a charging pad closest to a point below its coarse-navigation arrival position as a “best guess” (or “candidate”) for the designated target charging pad.also depicts a dashed-dotted circle at 1.5 meters, with a diagonal arrow labeled “~7%” indicating a likelihood that a coarse-navigation arrival position will be at or beyond this circle. Since 1.5 meters marks roughly a half-way point between charging pad-A and its immediate neighboring charging pads (e.g., charging pad-B), a UAV arriving at a coarse-navigation position beyond the 1.5 meters circle may be subject to mis-identifying a neighboring charging pad as charging pad-B, if identification is based solely on the closest charging pad to the UAV's coarse-navigation arrival position. The questions mark (“?”) in the neighboring charging pads of charging pad-A signify this potential ambiguity in identification. For the current example, and without any additional procedure for disambiguation, such a mis-identification may be expected to occur (or at least may be at risk of occurring) with an approximate likelihood of 7%, or approximately 7% of the time. Again, this value and the assumptions that lead to it are presented by way of example. However, the potential mis-identification by a UAV of a designated target charging pad due to limitations on the accuracy of the coarse-navigation mode applies at least conceptually to these and other possible statistical descriptions of navigational accuracy.

In order to aid or achieve disambiguation of identities of charging pads of a charging pad cluster, fiducial markers may be distributed among the charging pads to serve as distinctive visual cues to an imaging system of a UAV. One approach to deploying fiducial markers for this purpose is to provide a set of fiducial markers, each of which is uniquely visually identifiable from among the set, and each of which is placed at a precise (or nearly so) geolocation within a layout of a charging pad cluster. For example, each charging pad may have one or more uniquely identifiable fiducial markers associated with it. The UAV may be provided with information, such as a data table, that lists each charging pad and its associated fiducial markers. The UAV may then be able to cross-check observed fiducial marker identifications and charging pads against the data table in order to resolve mis-identifications of charging pads that may arise due to imprecise navigation, for example. In addition to the use of uniquely identifiable fiducial markers, and precisely (or nearly so) determining each of their geolocations (possibly as well as a specific orientation for each), this approach may also call for relatively regular or routine site maintenance in order to help ensure an accurate and/or up-to-date match between the site deployment and the data table that records it.

Example embodiments herein provide an alternative methodology for using fiducial markers to aid and/or achieve disambiguation of identities of charging pads of a charging pad cluster. Specifically, the alternative methodology involves using patterns or constellations of fiducial markers formed by their relative positions and orientations with respect to charging pads of a cluster to distinguish between the charging pads. In accordance with example embodiments, a respective sub-plurality of fiducial markers in the vicinity of each respective charging pad may form a different apparent constellation in relation to the respective charging pad. A reference map of the cluster that captures an aggregate of the respective constellations may be generated and provided to a UAV, and may then be used by the UAV for comparing constellations observed with its imaging system with “reference constellations” in the reference map. The UAV may thus use the comparisons to disambiguate between observed charging pads, and thereby correctly identify a designated target charging pad.

4 FIG.C 400 404 404 404 illustrates an example charging pad clusterwith example fiducial markers, in accordance with example embodiments. For purposes of discussion, the fiducial markersare represented somewhat cartoonishly as arbitrary shapes and with graphic patterns that serve as identifiable features. Thus, as shown the orientation of a fiducial may be distinguishable by its shape (e.g., having an asymmetry and/or a graphic pattern, such as graphic markers or numbers). The positions of the fiducial markersacross a layout of charging pads of a cluster may be configured according to a specific pattern, randomly assigned, or some blend of both, for example. Any arrangement should yield distinguishable constellations among the charging pads.

4 FIG.C 404 As shown in, there is some repetition among the fiducial markers. In the example illustrated, instances of identical fiducial markers appear in different orientations, but this need not be a requirement, so long as distinguishable constellations can emerge from the distribution of fiducial markers. However, using different orientations among identical fiducial markers may enhance the recognizability of constellations. In any case, the ability to distinguish different constellations even with the inclusion of at least some identical fiducial markers may make it possible to use at least some non-uniquely identifiable fiducial markers among a plurality of fiducial markers deployed across a charging pad cluster.

4 FIG.C In some embodiments, the fiducial markers may be “April tags.” As is known, April tags comprise graphical patterns of “low resolution” QR-like codes. Three square fiducial markers inrepresent an example of the same April tag in three different orientations, distinguishable by the orientation of the graphic pattern and/or of the square shape. For at least the purpose of this discussion, and possibly in at least one or more actual deployments, however, fiducial markers in addition to, or other than, April tags may be used.

400 402 404 400 402 400 4 FIG.C The illustrative depiction of charging pad cluster, including the fiducial markers, inmay be considered as representing a specified layout of charging padsand fiducial markers, or as representing a view or image taken from above of an actual deployment. In the latter case, an aerial image of charging pad clustermay be captured by the imaging system of a UAV that is hovering directly over charging pad-A at the center of the grid, for example. Such an image may be used to generate a graphical reference map of the charging pad cluster, which may then be provided to UAVs having missions that include flying to and landing on a designated target charging pad of the cluster.

400 400 400 400 In accordance with example embodiments, and as described in more detail below, a server in an infrastructure support network for UAVs may receive aerial observation data of charging pad clusterfrom one or more UAVs, and may process the aerial observation data to generate a graphical reference map of the charging pad cluster. The reference map may represent an optimization of aerial observation data in a form that may correct or compensate for possibly multiple perspectives and positions with respect to the charging pad clusterfrom which the aerial observation data may have been captured. One or more UAVs may have captured and processed one or more images to produce respective aerial observation data, and then transmitted (or otherwise provided) the aerial observation data to the server. The one or more UAVs may have captured the one or more images as part of specific missions to survey the charging pad cluster for the purpose of obtaining the images for generating the reference map. Additionally or alternatively, the one or more UAVs may have flown to the charging pad clusterand captured the images as part of operations to disambiguate charging pads in order to identify a designated target charging pad. Aerial observation data from such images may be used by the server to update, revise, and/or further optimize an existing reference map, as described below.

In some example operations, a UAV may start a mission from a particular “launching” charging pad. In an initial operational state prior to flight, the UAV may not have awareness of the charging pad identity, and may first ascend vertically to obtain an aerial image of the charging pad cluster in order to apply context-based navigation to determine its launching charging pad. The aerial image so obtained and/or processed information therefrom may be sent from the UAV to the server to provide a further image for updating and/or optimizing an existing reference map. In some operational scenarios, the server may use information in (or processed from) the aerial image obtained from the UAV's initial ascent to determine an identification of the UAV's launching pad, as well as the UAV's geolocation and orientation. The server may then provide the determined geolocation and orientation to the UAV in order to help initialize the UAV's operational state. This procedure is described in more detail below.

4 FIG.D 400 400 illustrates an example charging pad cluster with example fiducial markers as viewed from two different hypothetical perspectives, in accordance with example embodiments. Two panels are shown, panel (a) above and panel (b) below, each conceptually representing an aerial image of charging pad clusteras captured from a position above a different charging pad. In each panel, an “X” marks the charging pad above which the image was (hypothetically) obtained from. The perspectives as illustrated are somewhat exaggerated, and not necessarily intended to be to scale or otherwise accurate renditions of how the charging pad clusterwould actually appear from the positions above the marked charging pads.

In accordance with example embodiments, the server may receive aerial observation data from one or more UAVs. As described below, the aerial observation data may be used by the server to construct a map graph, which may then be optimized to generate a reference map of the cluster. In particular, the reference map may be used as a sort of virtual rendition, as viewed from above, of the constellation of fiducial markers in the vicinity of any given charging pad of the cluster.

5 FIG. 5 FIG. 500 400 502 502 402 402 500 500 502 502 500 402 402 is a representation of a reference mapof charging pad cluster, also depicting two portions of the map that represent reference constellations-A and-B of neighboring charging pads-A and-B, respectively. In accordance with example embodiments, the reference mapmay take the form of data that may be rendered graphically and/or computationally interpreted in a form equivalent or analogous to a graphical rendering. In, the map, shown on the right-hand side, is presented on a scroll-like background as a visual cue for the discussion that the figure illustrates a rendering of data. In some examples, the rendering could take the form of a synthesized image, for example. The reference constellations-A and-B, shown to the left of the map and similarly presented on scroll-like backgrounds, could be generated distinct subsets of the data of the reference map, or identified sub-regions of the reference map but not separately generated. In either case, they serve to illustrate what an idealized view from directly above the respective charging pads-A and-B may be expected to look like.

502 402 402 502 402 402 More particularly, reference constellation-A represents a constellation of fiducial markers at relative positions and orientations with respect to charging pad-A as that constellation would be expected to ideally appear from a position above charging pad-A. Similarly, reference constellation-B represents a constellation of fiducial markers at relative positions and orientations with respect to charging pad-B as that constellation would be expected to ideally appear from a position above charging pad-B. For the purposes of clarity in the figure, each reference constellation omits the neighboring charging pads. This omission could also be an intentional feature of some example embodiments that generate reference constellations separately from a reference map from which they are derived.

500 The qualification of a reference view or appearance as “ideal” may be considered a recognition that an actual aerial view or image from a UAV positioned above a charging pad and its associated constellation of nearby fiducial markers may differ to some degree from what appears in the reference mapor the reference constellations. Differences between an observed view or image from above an actual charging pad and its appearance in a reference map or reference constellation may be due to various factors. Examples of such factors may include differences in the UAV's orientation and/or altitude with respect to a charging pad cluster compared with an idealized orientation of the reference map; a missing, damaged, or obscured fiducial marker at the charging pad cluster that has not been accounted or corrected for in the reference map, and possible distortions of the idealized view of the reference map due to imperfect optimization of the aerial images used to generate the map. However, in accordance with example embodiments, the spatial distribution of fiducial markers across an arrangement (e.g., grid) of charging pads of a charging pad cluster may be configured such that alignment or matching of an observed constellation to a reference constellation need not necessarily be exact in order to achieve a threshold degree of agreement that may be interpreted as a confirmed match. In this sense, a confirmed match may tolerate some amount of disagreement.

6 FIG. 6 FIG. 4 FIG.B 402 400 402 502 502 402 402 is a conceptual illustration of an example operation for disambiguating charging pad identities using context-based navigation, in accordance with example embodiments. Continuing from the discussion above, it may be assumed that charging pad-A is a designated target charging pad for a UAV having a mission that includes flying to the charging pad cluster, and as described, the UAV may arrive initially at a coarse-navigation location that is above a ground point within approximately 3 meters of the designated target charging pad. For any particular instance, this arrival location could be closer to one of the neighboring charging pads than to charging pad-A.illustrates an example of how the UAV may use the reference constellations-A and-B to distinguish between charging pad-A and-B. The need to do so could arise if the UAV's coarse arrival location were above a ground point halfway between the two charging pads, or otherwise insufficiently close to either one to make a confident identification. Additionally or alternatively, UAVs could apply the disambiguation operation routinely, even if their coarse-navigation arrival locations are directly above a charging pad since, as discussed in connection with, the coarse-navigation location could be above the “wrong” charging pad ~7% of the time (depending on specifics of the coarse-navigation arrival location probability distribution, for example).

500 400 500 500 500 502 502 502 502 500 502 502 6 FIG. In accordance with example embodiments, a UAV may be provided by the server with a reference mapprior to arriving at a locale of charging pad cluster. For example, the UAV may be provisioned with the reference mapas part of preparing the UAV for a mission. In another example, the reference mapmay be transmitted to the UAV at some point after it begins its flight. If reference constellations of the charging pads are generated separately from the reference map(e.g., as separate data entities, for example), the UAV may also be provided with reference constellations-A and-B, possibly as well as reference constellations for one or more other charging pads of the cluster. Additionally or alternatively, the UAV may have a computational capability to generate reference constellations-A and-B (and others) from the reference map. Reference constellations-A and-B shown inmay represent the UAV's copies of these data as obtained or generated by any of the above, or possibly other, means.

400 400 Upon arriving at the locale of the charging pad cluster, or shortly thereafter, the UAV may acquire one or more aerial images of one or more charging pads nearby a ground point below its coarse-navigation arrival location. The UAV may use its imaging system to obtain one or more images. In one operational example, each image may capture a different one of the charging pads and an observed associated constellation of fiducial markers. Additionally or alternatively, more than one image may be acquired for at least one of the charging pads. As a further or alternative example, the images may correspond to different portions of a continuous video image acquired (or as it is being acquired) while the UAV hovers in place and/or moves about above the charging pad cluster. As still another example, each of one or more images of one or more charging pads may be acquired from respective locations above each of the one or more charging pads. In yet another example, a single image from the UAV's coarse-navigation arrival location may suffice if the UAV's altitude is high enough to render any differences in perspective due to horizontal translations to be negligible in their effect on matching observed constellations to reference constellations.

Which one or more of the above modes of image acquisition is used may depend on operational circumstances, such that a UAV may decide which one best accommodates any particular disambiguation task instance. Additionally or alternatively, a sequence of disambiguation procedures, including what images should be acquired and how, may be more fully specified ahead of time. It should be understood that any of the above or other alternatives may accommodate, or be accommodated by, the principles of context-based navigation as described herein.

502 502 402 402 602 602 6 FIG. 6 FIG. For purposes of the present discussion, only reference constellations-A and-B and aerial images of charging pads-A (the designated target charging pad) and-B are considered. In practice, the disambiguation operation may need to consider the designated target charging pad and more than one other neighboring or nearby charging pads. As represented in, observed constellations-A and-B are captured in respective images acquired by the UAV's imaging system. The depictions of the images shown inmay be considered representations of separate image captures in an image plane (e.g., pixel array) of an imaging system, or different regions of a single image capture in the image plane. In some examples, the images may be still images, and in some examples they may represent frames of one or more video images. These examples are not meant to be mutually exclusive, but rather may be different options or modes of operation of the imaging system.

502 502 400 4 FIG.A The images may be acquired by any one or more of the example operations described above. Each image depicted is at an angle with respect to the reference constellations-A and-B to signify a possible orientation of the UAV and/or the camera component of its imaging system with respect to a reference direction of the charging pad cluster(e.g., as define by the X and Y axes of). For purposes of clarity in the figure, neighboring and/or nearby charging clusters are omitted from the images. In actual operation these charging pads may be visible in one or more of the images. However, their presence in an image would not be expected to change the appearance of the observed constellations.

602 602 502 502 402 502 502 602 602 6 FIG. In accordance with example embodiments, the UAV (or a controller or other processing entity thereof) may compare the observed constellations-A and-B of the acquired images with the reference constellations-A and-B in an attempt to positively identify the observed charging pad in each image. In this way, the UAV may be able to positively identify charging pad-A as its designated target charging pad, and then proceed to land on it (e.g., assuming landing on the charging pad is specified in its mission). Confirmation of the identity of an observed charging pad may be carried out by a constellation-matching operation and charging pad identity disambiguation, indicated by the so-labeled block at the center of. As shown by way of example, inputs to this block may be the reference constellations-A and-B and the UAV images of the observed constellations-A and-B. The output of the operation is the charging pad identity confirmation, as indicated by the so-labeled data item icon.

In making a match, the matching operation carried out by the UAV (or its controller, for example) may compensate for differences in the orientations of the acquired images and the reference constellations. In accordance with example embodiments, the matching operation may utilize, or be implemented as, a computational pattern-matching algorithm. For example, pattern matching may function by treating a charging pad and the fiducial markers of an associated constellation as vertices of a graph, and thereby seek to align vertices in an observed constellation with vertices in a reference constellation. For each attempted match, a goodness-of-fit statistic may be computed and compared with a threshold. A fit that exceeds the threshold may be considered a match. If there is more than one potential match, then one that maximally exceeds the threshold may be deemed the positive match.

In another example, the matching operation may utilize, or be implemented as, a machine-learning model or program. For example, an artificial neural network (or similar machine-learning model) may be trained to recognize matches among two or more input graphical patterns. Such a model may then be used to predict a confidence metric for a match between a given reference constellation and one or more observed constellations. The confidence metric may then be used in a similar manner to a goodness-of-fit statistic described above.

It will be appreciated that there may be other programs, algorithms, and/or procedures for determining a match between a given reference constellation and one or more observed constellations. In accordance with example embodiments, matching for disambiguation of charging pad identities and/or identity confirmation may be equivalently or analogously achieved by determining a match between a given observed constellation and one or more reference constellations. A positive identification of a charging pad may be derived from either “direction” of matching.

502 402 500 As with the various modes of image acquisition, there can be various sequences of operations for determining a match between reference constellation-A of charging pad-A as it appears in the reference mapand one of the observed constellations in order to confirm a positive identification of the UAV's designated target charging pad. The direction of matching—i.e., matching a reference constellation to one or more observed constellations, or matching an observed constellation to one or more reference constellations—is one example of variation. Other, non-limiting examples are described in the following.

4 FIG.B 402 In one example, a charging pad closest to a ground point below a UAV's coarse-navigation arrival location may be selected as an initial guess for the UAV's designated target charging pad. This may be based on a known probability distribution for coarse-navigation arrival positions that peaks at a designated navigational target, as illustrated in, for example. Initially, the UAV may acquire one or more images of just the selected charging pad, and may then check if an observed constellation in the one or more images matches a reference constellation of the designated target charging pad (e.g., charging pad-A in the above example). If a match is found, a confirmation may be made effectively on the first try. If a match is not found, the UAV could acquire one or more images of a next-nearest charging pad, and again check if the constellation of the image(s) matches the reference constellation. If not, the process could repeat for as many observed charging pads as necessary until a match is found.

In an alternative to the above example, if the observed constellation of initial selected charging pad does not match the reference constellation of the designated target charging pad, further matching attempts against other reference constellations may be made until a match is found. This match would not identify the designated target charging pad, but would confirm an identity of the charging pad closest to the ground point above the UAV's arrival location. This information could then be used to inform or optimize a search strategy for selecting a next guess of observed charging pad to image and check for a match. Again, this process could repeat until a positive match of the UAV's designated target charging pad is determined.

In still another example, the UAV may acquire one or more images of several—e.g., four or five-charging pads nearby, and including, an initial best guess (e.g., one closest to a ground point below the UAV's arrival location). These could be obtained by the UAV moving about horizontally to locations above each of the charging pads to be imaged. Alternatively, as described above, if the altitude of the UAV's coarse-navigation arrival location is high enough, a single one (or a few) images from the arrival location may capture all of the nearby charging pads in the image field of view (FOV) with negligible perspective distortion across the image. For either image acquisition procedure, a reference constellation of the designated target charging pad could be compared against respective constellations associated with each of the charging pads in the one or more images to find out which one produces a match. The best match could then be used as a confirmation of identity of the designated target charging pad.

As still a further example, images of multiple observed constellations and multiple reference constellations may be input to the matching operation, such as a pattern matching algorithm and/or machine-learning model (e.g., trained to recognize matching patterns and output statistical confidences of detected matches). The output of the matching operation could confirm identities of one or more charging pads associated with the observed constellations. The UAV could then determine which of the confirmed identities corresponds to its designated target charging pad.

As noted, the above techniques and procedures for determining matches between observed constellations and reference constellations in order to disambiguate among multiple observed charging pads are not intended to be limiting. Rather, all of these and others illustrate that there may be numerous ways to implement the principle of context-based navigation, and demonstrate the flexibility and versatility of using distinguishable constellations of fiducial markers in the vicinities of observed charging pads to confirm identities of the observed charging pads. Applied by (or for) UAVs that may otherwise have only a coarse-navigation mode that provides insufficient accuracy for the UAVs to make confident identifications based on coarse-navigation arrival positions, context-based navigation thereby enables UAVs to fine tune their navigation capabilities. As also noted, a general degree of tolerance of context-based navigation to imperfect matches of observed and reference constellations may reduce and/or relax possibly stringent requirements on charging pad cluster site maintenance as it applies to upkeep of fiducial markers.

In further accordance with example embodiments, a UAV at a coarse-navigation arrival location may have a real-time communicative connection with a mission operations server in the infrastructure support network, such that some or all of the matching operations described above may be carried out by the mission operations server (or other support system of the network). As an example, the charging pad cluster at the locale of the UAV's coarse arrival location may deploy a WiFi network to which the UAV may connect, as described above, for example. The WiFi network may have a broadband interface (e.g., via a gateway router, for example) providing a link to the mission operations server. Additionally or alternatively, the UAV may have its own broadband interface (e.g., SIM card) that provides a communicative link to the mission operations server. For either of these (or other) communication scenarios, the UAV may transmit in real-time (or near real-time) some or all of its observed images to the mission operations server. The mission operations server may then carry out the analysis described above to determine a confirmed identity of the UAV's designated target charging pad, and then transmit the confirmation information back to the UAV. The UAV may then proceed based on the confirmed identity.

In some server-based implementations, the mission operations server may maintain the requisite reference maps and reference constellations, such that UAVs might not necessarily be required to carry (or be provisioned with) their own copies. The UAVs may then only need to transmit in real-time (or near real-time) their images of observed charging pads. In other implementations, UAVs may carry (or be provided with) reference maps and reference constellations, and may thus transmit in real-time (or near real-time) their images of observed charging pads, as well as any reference constellations needed for matching (e.g., according to one or more of the above techniques). These are just examples of context-based navigation in which a mission operations server may carry out various aspects of matching and disambiguation.

While example embodiments of context-based navigation have been described above in terms of charging pads and charging pad clusters, the principles may be applied, adapted, and/or extended to other types of navigation targets and configurations thereof. An example of a different usage scenario of context-based navigation using fiducial markers in a manner substantially similar to that described above involves automatic loading facilities used in UAV delivery operations. In an example deployment, delivery UAVs may pick up or retrieve payloads or packages from automated payload dispensers or loaders that are distinct from charging pads, and possibly configured at facilities separate from charging pad clusters. Such an operational scenario for payload pick-up may be used as an alternative to payload pick-up at charging pads, or as an additional mode that augments charging-pad pick-ups.

In accordance with example embodiments, multiple automated payload loaders, sometimes referred to as “autoloaders,” may be deployed in arrangements similar to charging pads at a cluster. For example, a cluster of autoloaders may be configured in a layout at a warehouse or other storage facility. Some or all of the autoloaders may be, or appear to be, sufficiently alike as to make them potentially indistinguishable (or nearly so) to an imaging system of a UAV. A UAV with a mission plan that includes flying to a designated target autoloader at such a facility may face the same or similar navigational challenges as those described above in connection with charging pads and charging pad clusters. A plurality of fiducial markers may thus be distributed across a layout of autoloaders in the same or similar manner as that described for charging pad clusters. With such an arrangement, the principles of context-based navigating using observed and reference constellations of fiducial markers may be straightforwardly applied to clusters of autoloaders. A UAV may thus be able to distinguish between and/or disambiguate autoloaders, and thereby confirm an identification of a designated target autoloader.

It should be understood that confirmation of the identification of a designated target autoloader by a UAV need not necessarily be followed by the UAV landing on the target autoloader (or charging pad, for that matter). For example, in some UAV delivery scenarios, a UAV may hover above an autoloader and lower a tether to retrieve a payload or package, then reel the tether back in to secure the payload to the UAV (the same operation could apply for a payload or package retrieved from a charging pad). Other remote retrieval techniques that do not necessarily require a UAV to land may be used as well.

As another example of a different usage scenario of context-based navigation, charging pads and/or autoloaders may be equipped or configured with unique identifiers, such as unique April tags or other forms of fiducial markers, such that under expected operational conditions the charging pads and/or autoloader are visually distinguishable from one another by an imaging system of a UAV. In this arrangement, UAVs may be able to distinguish between, or disambiguate, charging pads and/or autoloaders much or most of the time—e.g., 80-90% of the time possibly on the basis of the unique markers of the charging pads alone. But the arrangement of uniquely identifiable charging pads may also serve as its own form of constellations, even without necessarily including separate fiducial markers in the manner described above. Thus, in accordance with example embodiments, context-based navigation may also be achieved as described above by using observed and reference constellations of uniquely identifiable charging pads (and/or autoloaders). As with the approach using separate fiducial markers, uniquely identifiable charging pads (and/or autoloaders) as constellations may provide the same or similar tolerance or resilience to the ability of UAVs to make positive identifications when faced with markers that become damaged, degraded, and/or obscured by detritus, for example.

These are just two examples of additional or alternative applications of context-based navigation using observed and reference constellations of fiducial markers. It should be understood that the principles may be adapted, extended, and/or applied to other usage scenarios as well.

500 300 310 700 712 310 3 FIG. 7 FIG.A As described briefly above, reference maps of charging pad clusters, such as reference map, may be generated by a server in a UAV infrastructure support network, using aerial observation data collected by one or more UAVs in the course of routine flights to clusters and/or designated survey flights to clusters. In accordance with example embodiments, reference maps may be generated and maintained as part of an asset management system that may also perform tracking (e.g., establishing and maintaining status records, etc.) of ground-based assets of charging pad clusters—e.g., charging pads, fiducial markers, and/or autoloaders—and/or other UAV ground-support facilities. In addition to one or more servers, an asset management system may include an asset database for storing both reference maps and asset records. By way of example, and referring again to the UAV systemof, an asset management system may be implemented in, or be part of, a central dispatch systemor other facility of a UAV system. An example of such an arrangement is illustrated in, which depicts an asset managerand an associated asset databaseas part of the central dispatch manager.

7 FIG.A 700 700 700 700 In the example of, asset managermay include one or more servers or other computing systems configured for carrying out tracking of ground-based assets and generation and maintenance of reference maps. As noted, the associated asset database may be configured for storing and maintaining ground-based assets and reference maps generated by the asset manager. The asset managermay also include, or be communicatively connected with, a communications interface for communications with UAVs. For example, asset managermay be communicatively connected with a cellular wireless interface to a cellular network that operates according to 3G, LTE, 4G, or 5G, among other possible protocols.

100 100 400 400 400 100 701 1 FIG.A 7 FIG.A 4 FIG.D 7 FIG.A Asset management as carried out by an asset management system of a UAV system, including reference map generation and asset tracking, may be understood in terms of example operation involving just a single representative, example UAV(from) in. By way of example, UAVis depicted as stationed (e.g., hovering) above the charging pad marked with an “X” in charging pad clusterof, panel (a). For the sake of brevity in, only four charging pads and two fiducial markers of clusterand show (ellipses indicate the further expanse of cluster). The aerial geolocation and orientation of UAVare designated as its pose. As used herein, the term pose may specify the location and position of a UAV, including its geographic coordinates, altitude above ground level (sometimes referred to as “AGL”), and angular orientation (e.g., pitch, roll, and yaw).

100 702 100 100 100 202 2 FIG. In accordance with example embodiments, geolocation may be determined from GPS coordinates, as indicated in a communication link (lightning bolt) between UAVand a GPS satellite, although other geolocation techniques may also be used. Altitude may be determined in one or more of a variety of ways. For example, it could be directly measured if the UAVis equipped with a measuring device such as an altimeter, or RADAR or LIDAR for measuring line-of-sight distances. As another example, the UAVcould triangulate distance by shifting its position horizontally by a specified distance (while maintaining altitude) and measuring a resulting parallax (angular) shift of an object on the ground. As still another example, the UAV could compare a known actual linear size of asset on the ground with its apparent (observed) size (e.g., in a real-time image). Other techniques may be possible as well. UAVmay determine its angular orientation from an inertial measurement unit, such as IMUshown in, for example.

701 100 100 The components of posemay be specified to locate the origin and angular orientation of a UAV coordinate system (tied to UAV) within a pose coordinate system (e.g., geographic latitude and longitude, altitude, and geographic or magnetic North). As such, a known position and orientation of an imaging system of UAV, specified with respect to the UAV coordinate system, may be used to transform any image captured in an imaging plane of the imaging system to the pose coordinate system, or to any desired orientation therein.

100 400 400 400 703 400 100 701 400 701 7 FIG.A In accordance with example embodiments, UAVmay use one or more images of all or a portion of clusteracquired from one or more pose positions above the clusterto determine vector positions to one or more assets of the cluster. The example illustration ofdepicts six representative vectorsto four charging pads and two fiducial markers of clusterthat UAVhas determined from its pose position; there could be additional determined vectors as well to other ground-based assets of cluster. Each vector may specify a location or positon of a respective ground-based asset reckoned with respect to the UAV coordinate system (e.g., as x, y, and z vector components), which is positioned according to the pose position.

400 100 400 100 100 Vector components of any given ground-based asset may be determined from an image or snapshot of the clusterin one or more of a variety of ways. For purposes of illustrating just one example, it may be assumed that the image plane of an imaging device of the UAV is parallel to the x-y plane of the UAV coordinate system, which may in turn be assumed to be normal to yaw-axis of UAV. It may be further assumed that the z-axis of the UAV coordinate system is parallel to the yaw-axis. Further assuming that clusteris configured on a level (horizontal) plane, the altitude of the UAVmay be taken as the z-component of a position vector of the given asset. The x-y position of the given asset in the image plane may then be converted by straightforward geometry to x and y vector positions in the UAV coordinate system. If the image plane is not parallel to the x-y plane, and/or if UAVis not stationed horizontally when the image is captured, then one or more additional rotational translations may be necessary to determine the vector components from the image data and pose position. However, all such transformations may be achieved with routine geometric operations.

100 400 100 100 100 In accordance with example embodiments, UAVmay capture multiple images from different pose positions, and determine more than one vector position for any one or more assets of the cluster. For example, UAVmay obtain images from multiple pose positions if it is carrying out a designated survey mission. As another example, images from more than one pose position might be necessary if UAVuses parallax to determine altitude. As noted, another technique for determining altitude is to compare an apparent size of an asset in the image plane with a known size of the asset. For example, charging pads may typically have known linear dimensions. Thus, comparison of the observed size in an image with the known size may yield a line-of-sight distance, which could correspond to a z-component if UAVis hovering directly over the charging pad, or converted to a z-component via additional geometric operations otherwise.

100 400 100 400 100 Also in accordance with example embodiments, the imaging device of UAVmay be a video camera or device that produces video frames of cluster(or of any other field of view of the video device). In this arrangement, video frames may be processed by the video device and/or by a processing operation of UAVto effectively generate still-image snapshots of clusterfrom sequences of frames captured from the same (or nearly the same) pose positions. For example, assuming the video device produces 24 frames per second, a snapshot might be generated every 10 seconds of constant (or nearly constant) pose position by processing—e.g., averaging-successive sequences of 240 frames. UAVmay also record the pose and time of each snapshot.

100 100 100 400 In further accordance with embodiments, UAVmay identify some or all assets in each snapshot image, and determine vector positions to some or all of the identified assets. To the extent that the same asset is identified in multiple snapshot images, UAVmay thus determine more than one vector to that asset. If at least some of the multiple snapshots correspond to different pose positions, then at least some of the vectors of the asset may represent a view of the asset from different perspectives of the imaging system of UAV. This arrangement of multiple vectors for the same asset may apply to more than one asset of cluster.

100 In practice, techniques used by UAVto identify charging pads in a cluster may differ from those used to identify fiducial markers (or possibly other forms of ground-based assets). For example, identification of charging pads in multi-frame snap shots may use a different number of frames per sequence than identification of fiducial markers. There may also be differences between algorithms used to identify charging pads and those used to identify fiducial markers. As such, in some examples, identification of assets, and determinations of vectors to them, may be carried out separately for charging pads and fiducial markers.

100 In some examples, identifications of assets assigned by UAVmay only identify a type of asset—e.g., a charging or a fiducial marker—but might not include unique (or presumed unique) identifications. For example, charging pads might not have any distinguishing identifiers, but rather may appear identical to one another. And as described above, the use of constellations of fiducial markers for fine-mode navigation may relax any requirement that all fiducial markers of a cluster carry unique identifications, so long as no two or more identically-identified fiducial markers are situated closer than some distance and/or relative position threshold of each other. As described below, the asset manager, using aerial observation data transmitted from UAVs, may generate reference maps without necessarily requiring exact knowledge of asset identifications.

100 700 705 705 400 100 705 400 400 100 705 7 FIG.A In accordance with example embodiments, UAVmay send its above-described observations, measurements, and determinations to the asset managerin the form of aerial observation data, as indicated in. The aerial observation datamay include one or more determined vector positions to each of one or more ground-based assets of cluster, a pose position from which, and time at which, each vector position was determined, and an asset identification associated with each of the one or more determined vector positions. In further accordance with example embodiments, UAVmay generate aerial observation dataon a continuous basis, periodically, and/or episodically while carrying out aerial operations above clusteror the locale of cluster. For example, UAVmay generate aerial observation datain a sequence of subsets every 10 seconds, corresponding to 10-second snapshots, as described above. It should be understood that subsets could correspond to different intervals, as well, and subset intervals need not necessarily be the same length.

100 700 705 100 700 100 700 100 100 UAVmay use a wireless network connection with the asset managerto transmit the aerial observation data. As described above, such a wireless network connection (depicted as a lightning bolt between UAVand the asset manager) could be provided by a cellular wireless network operating according to one or more known protocols. In some examples, UAVmay transmit aerial observation data subsets to the asset managercontinuously, as the subsets are generated. Additionally or alternatively, UAVmay transmit aggregations of subsets periodically or episodically. As still another example, UAVmay choose from among multiple, different transmission modes according to operating conditions or scenarios.

705 705 Advantageously, data components of aerial observation datamay be constructed and/or formatted relatively small in terms of required transmission size, particularly compared with typical data sizes and volumes of still and/or video images (and numbers of video frames). As such, cellular wireless transmissions of aerial observation datamay incur relatively low cost in terms of bandwidth and/or monetary charges. This may, in turn, help keep operational costs of asset management relatively low.

700 705 100 400 700 705 707 712 7 FIG.A In accordance with example embodiments, the asset managermay use aerial observation datareceived from UAVto create a new reference map, or update an existing reference map, of cluster. The asset managerwill use received aerial observation datato create a new asset record, or update an existing asset record, for each distinct asset represented in the aerial observation data. Reference map for known clusters of a UAV system, as well as asset records for known ground-based assets (charging pads and fiducial markers), may be stored as asset tracking datain the asset database, as indicated in. Example techniques for generation of reference maps are described below.

7 FIG.B 7 FIG.B 705 707 705 705 illustrate example formats of aerial observation dataand asset tracking data, in accordance with example embodiments. As shown, aerial observation datamay include an UAV ID of the UAV that generated the data, and a Cluster ID of the charging cluster to which the data refer. The UAV ID “7E1” and Cluster ID “5094” inare arbitrary examples for purposes of illustration. Aerial observation datamay be constructed by grouping determined position vector positions and their associated asset IDs according to a UAV pose from which, and time at which, the observations used in the vector determinations were obtained.

7 FIG.B 1 2 3 1 2 2 1 1 2 3 11 11 11 12 12 12 13 13 13 2 3 1 2 1 2 3 2 2 4 3 By way of example in, three UAV poses, Pose, Pose, and Pose, at times T, T, and T, are shown. Each pose and time could correspond to an average snapshot from a sequence of video frames, or a single still image at the associated time. At Pose, assets identified as ID, ID, and ID, were observed, with determined vectors (x,y,z), (x,y,z), (x,y,z), where the first index of the vector components refers to the pose ID, and the second index refers to the asset ID. Similarly labeled groups of asset IDs and vectors are shown for Poseand Pose. Vertical ellipses indicate possible additional poses and associated data. As an illustration of how a given asset may be observed from different poses, assets IDand IDare shown to have been observed from poses,, and. Similarly, asset IDis shown to have been observed from poses land. Asset IDis shown to have been observed from pose.

707 709 711 709 709 709 7 FIG.B The example asset tracking datais shown inas including asset recordsand reference maps. By way of example, each asset recordis shown as including an asset ID of the tracked asset, a cluster ID of the charging cluster to which the asset belongs, and map coordinates of the asset in a reference map of the asset's cluster. For each asset, the asset recordalso includes historical data, recording historical values of vector positions of the asset as determined from a recorded pose position at an associated recorded time. In this example, the cluster ID of an asset recordmay be used as a pointer to a current reference map of the cluster, and the map coordinates of an asset may refer to coordinates of the asset in the current reference map. As described below, the historical data of a given asset, and of any other assets in the given asset's cluster, may be applied with any newly-received aerial observation data for assets in the cluster to update the current reference map.

7 FIG.B 1 2 9 5094 6215 705 705 709 1 709 91 In, assets with asset ID,, . . . ,, . . . are shown as illustrative examples. The cluster IDsandare also just illustrative examples. In some examples, the asset IDs in the asset records may not necessarily correspond exactly to the asset IDs used in the aerial observation data. This may be the case, for example, if the asset IDs in the aerial observation dataare used only to tag assets as distinct input entities of map graph optimization, while the asset IDs in the asset recordsare applied as unique identifiers in a reference map determined from map graph optimization. The indices of the map coordinates (e.g., xml, ym) in the asset recordscorrespond to the asset IDs, and the indices of the historical data (e.g., time) correspond to the asset ID and history sequence index, respectively. Other indexing schemes could be used as well.

711 In accordance with example embodiments, reference mapsmay include a map record for each current reference map. Each map record may include a cluster ID of the cluster to which the reference map applies, a timestamp indicating when the reference map was created or updated, and a pointer to a file or other data structure where a graphical representation of the map may be stored. The stored graphical representation of a reference map may be retrieved and provided to a UAV for navigation to the cluster associated with the map, as described above, for example. The reference map may also be retrieved as part of an updating procedure, as described below.

In accordance with example embodiments, a reference map of a cluster may be generated by using vector positions and poses of assets from aerial observation data of the cluster to construct a connected graph, referred to herein as a map graph, and then to optimize the map graph to determine positions of the assets rendered in the form of a map. In the map graph, the assets-charging pads and fiducial markers—and one or more poses from which vectors were determined may define nodes, where the positions of the nodes correspond to apparent positions of assets in a common reference plane. The apparent positions may be determined by straightforward geometric transformations of vector positions with respect to the associated pose (or poses) to the common reference plane. Apparent distances between the nodes may define connecting edges of the map graph. The edges may further be conceptualized as elastic connections, such as springs, with spring constants having strengths inversely related to measurement uncertainties of the apparent distances. Thus, the smaller the uncertainty, the stronger the spring constant, and vice versa.

Optimization of the map graph adjusts the relative distances between nodes, thereby adjusting the apparent positions of the nodes toward the actual positions of the associated assets. The spring constants act to enforce greater resistance to changing relative positions between pairs of nodes connected by edges with low measurement uncertainties (i.e., strong spring constant), and to offer less resistance to changing relative positions between pairs of nodes connected by edges with higher measurement uncertainties (i.e., weaker spring constant). In practice, measurement uncertainties may be due to a number of factors. Non-limiting examples may include GPS measurement accuracy (itself affected by signal strength and/or interference, for example), positional stability of UAVs during acquisition of observational data, weather conditions, poses of UAVs, and ground conditions that may impact visibility to, or distinguishability of, assets.

A reference map may be generated in this manner from the optimization of a map graph constructed using only UAV observations and the associated poses from which the observations were made. The technique may be thought of as a sort of bootstrapping using relative positions between the UAV or UAVs and the assets. A class of analytical algorithms of this form is sometimes referred to as simultaneous localization and mapping, or “SLAM.” In some implementations of SLAM or SLAM-like techniques, edges may be configured only between node pairs consisting of one “pose node” and one “asset node” and between pairs of pose nodes. However, other map graph configurations may be used.

In principle, a map graph can be constructed, and a resulting reference map generated, from observations of just a single UAV at one pose position above a cluster. That is, the analytical expression of the graph optimization may be formulated from just one set of vectors determined from a single observation point. In practice, however, multiple sets of vector determinations from multiple poses may yield superior accuracy of results. Including observations from multiple UAVs obtained at different times may improve accuracy even more. However, for purposes of illustrating SLAM or similar techniques, only a single, more limited set of observations may be considered.

8 FIG. 7 FIG.A 8 FIG. 8 FIG. 8 FIG. 400 701 700 is a conceptual illustration of apparent positions of ground-based assets as viewed by a UAV, in accordance with example embodiments. In this illustration, actual positions of charging pads and fiducial markers of charging clusterare shown in gray, while the apparent positions determined from observations are shown in black, as indicated. By way of example in this illustration, the apparent positions are assumed to have been observed from just one UAV pose, such as poseshown in. As noted, aerial observations may be obtained from multiple poses and multiple UAVs. For any particular observation or set of observations by a UAV, the actual positions may not necessarily be known to the UAV when the observations are made. For example, if the UAV is carrying out a first-time survey of a new charging pad cluster, it may not have a reference map. In, the actual positions serve as reference positions to illustrate how apparent positions may differ therefrom. The apparent positions of the assets inrepresent reconstructed positions determined by the asset managerusing aerial observation data to transform the vector determinations of the UAV to apparent ground positions, as described above. The evident misalignment of actual and reconstructed apparent positions depicted inmay include possible exaggerations and arbitrary directional shifts that serve to illustrate the concept.

9 FIG. 9 FIG. 8 FIG. 901 901 901 701 901 901 901 Next,is a conceptual illustration of a map graphand optimization thereof, in accordance with example embodiments. In the example of, the map graphreproduces the apparent positions of the assets of. For the sake of clarity in the figure, the apparent positions of the assets in the map graphcorrespond to just a single pose (e.g., pose). In general, however, a map graph may include observations made from multiple different poses, possibly from multiple UAVs. Thus, map graphalso includes several poses, represented as black dots, even though the apparent positions from just a single pose are shown in this example illustration. Connecting edges, represented as springs, are shown between the poses and the assets, and also between some pairs of poses. The edges represent constraints between pairs of nodes, and the “spring constant” corresponds to the strength of the constraint. The map graphthus forms a partially connected graph in this example. A legend in the lower left of the map graphindicates the symbols for the nodes and edges.

In accordance with example embodiments, the map graph may be formulated analytically as a nonlinear least-squares optimization problem, which may seek to minimize a sum of terms expressed, in a general form, as

j j l i i i i i 902 901 902 500 9 FIG. subject to l≤x≤u, where ρis a loss function, f(·) is a cost function, and land uare upper and lower limits of the parameter x. In accordance with example embodiments, the minimization may be solved by a graph optimizer. The graph optimizer may be a nonlinear least-squares graph solver implemented computationally as computer-readable instructions carried out by a computing device or system, for example. Conceptually, the map graphmay be considered input to the graph optimizer, and the reference mapmay be considered output, as indicated by the broad arrows in.

8 9 FIGS.and 901 500 901 500 500 901 901 901 901 901 The depictions ofmay suggest or give the impression of a one-to-one relationship between the number of assets represented as nodes in the map graphand the number in the generated reference map. While this could be the case in principle, in practice there may be more assets represented in the map graphthan are ultimately identified in the reference map. This is because the reference mapshould, if properly generated, include only as many assets as actually exist in the observed cluster, while the map graphis constructed from what may be an aggregate of observations of a cluster. The aggregate measurements may thus correspond to possibly multiple vectors to each of one or more observed assets from possibly multiple poses and/or multiple UAVs. As such, there may be multiple instances of a given asset represented as nodes in the map graph, each with a different apparent node position. There may also be multiple pose nodes in the map graph—which may be a reason for multiple vectors for some of the assets. In this sense, the map graphmay contain both discrepant (apparent) positions of the assets with respect to their true position, as well as possibly disparate positions among multiply-observed assets. It should be understood that the particular form of the map graphis an example that serves as a conceptual illustration of SLAM or SLAM-like optimizations.

901 901 400 In accordance with example embodiments, the graph optimization procedure operates to effectively resolve both the discrepancies from true positions, as well as position disagreements among any multiply-observed asset. To the extent that solved positions of any multiply-observed asset in the map graphdo not align perfectly, the solved map graphmay yield clustering of solved reference map positions for that asset. In this case, some form of average (e.g., weighted average) of clustered positions may be determined and taken as the solved, true map position of the multiply-observed asset. Other methods for determining a solved position of a cluster of positions are possible as well. Thus, example embodiments may further include cluster analysis for identifying in clusters of positions, determining which positions may belong to an identified cluster, and computing a best solution to an asset position from a cluster of positions. It should be noted that use of the term “cluster” and the like in this context is distinct from (and should not be confused with) its use in connection with charging pad “clusters” (such as cluster).

901 901 500 The particular edges included in the map graphare illustrative, and shown by way of example. Analytically, the form of map graphimposes constraints only between certain pose nodes and certain asset nodes, and between certain pairs of pose nodes. Thus in this example, the optimization is not subject to constraints between asset nodes. In addition to solving for positions of the asset nodes in the reference map, the optimization may also yield solutions to the poses—e.g., solved positions and orientations that may be more accurate than the self-reported poses provided by UAVs in their aerial observation data. It should be understood that any given graph map constructed from reported aerial observation data may include different nodes and edges, possibly including edges between asset nodes. In any given analysis scenario, the nodes and edges included may depend on the specific aerial observations used and how the optimization problem is formulated in detail.

700 When a reference map is generated, it may either be a first-time map for a charging pad cluster, or a new map for a known cluster for which a previously generated map exists. Correspondingly, any given asset in a generated reference map may be a new asset not previously tracked by the asset manager, and for which no prior asset record exists, or may be a known asset of the UAV system with an existing asset record. A new asset may be determined either when its corresponding map is a first-time map, since a first-time map will typically include previously untracked assets, or when the asset has been newly added to an known cluster for which there is previously existing map and known, tracked assets. There may be other special or less typical situations which give rise to new maps and/or new assets. In any case, the asset managermay adapt its operations of map graph construction, reference map generation, and asset tracking depending on whether the generated map is or will be new, and/or whether any of the assets of reference map are new.

700 711 707 700 700 In accordance with example embodiments, the asset managermay determine that there is no existing map for a charging pad cluster by, for example, an absence of a reference map for the cluster in reference mapsof the stored asset tracking data. The asset managermay make this determination using a cluster ID in received aerial observation data for the cluster. This scenario could correspond to the asset managerreceiving aerial observation data from one or more UAVs carrying out aerial surveys of a newly-deployed charging pad cluster, for example. Other scenarios are possible as well.

700 700 711 707 700 709 707 For first-time reference map generation for a given charging pad cluster, the asset managermay wait for a threshold amount of aerial observation of the given cluster to be received before construction a map graph and generating the reference map in a manner described above. This could entail receiving and storing aerial observation data from multiple UAVs over multiple survey flights to the given cluster. Once the first-time reference map is generated, the asset managermay store it in reference mapsof the asset tracking data. The asset managermay also create an asset record for each asset identified in the first-time reference map, populating the fields accordingly and storing the records in assert recordsof the asset tracking data.

700 711 707 700 700 700 In accordance with example embodiments, for an alternative scenario, the asset managermay determine that there is an existing map for a charging pad cluster by, for example, finding a reference map for the cluster in reference mapsof the stored asset tracking data. Again, the asset managermay make this determination using a cluster ID in received aerial observation data for the cluster. This scenario could correspond to the asset managerreceiving aerial observation data from one or more UAVs carrying out aerial surveys of a known and mapped charging pad cluster, for example. Additionally or alternatively, this scenario could correspond to the asset managerreceiving aerial observation data from one or more UAVs making routine flights to the charging pad cluster. Other scenarios are possible as well.

700 700 700 In further accordance with example embodiments, when generating a reference map for a known charging pad cluster with an existing map, the asset managermay aggregate any newly-received aerial observation data with historical data stored in the asset records of the assets previously identified as belonging to the known cluster. In this way, a new map graph may be constructed from aerial observation data that accounts for both the new and historical observational data. Correspondingly, the generated reference map will also account for the aggregate observational data. When including historical data in the analysis process, the asset managermay omit or exclude historical data that are older than some threshold age or look-back time. Alternatively, the asset managermay cull the historical data from time to time (e.g., when generating new reference maps) to discard data older than a threshold age, such that the historical data only includes data “younger” than the threshold, making all of it suitable for inclusion with newly-received aerial observation data.

Application of an age threshold to exclude or omit historical data may act as a sliding window in map graph construction and reference map generation. The effect may be viewed as a sort of finite impulse response (FIR) filter on the analysis process. An alternative embodiment could be to include all historical data without any age thresholding. In this case, the effect may be viewed as a sort of Kalman filter on the analysis process. Either approach may be used, though the FIR filter approach may be more suited to excluding possibly obsolete and/or stale historical data.

700 711 Also in accordance with example embodiments, once a new reference map is generated for a known cluster, the asset managermay compare the new map with the currently existing map stored in reference maps. The comparison may be used to identify differences between the two maps that may represent changes to the physical deployment of the known charging pad cluster. For example, a newly-added cluster asset may be identified by the comparison. As another example, an absence of a known asset, and/or changing in position of a known asset may be discovered in this way. Other types of changes or modifications may also be discovered.

700 For any discovered changes that meet some predefined or predetermined threshold or criteria, further actions by the asset managermay be triggered. For example, discovery of a new charging pad may trigger a process to confirm agreement with a prior planned maintenance upgrade to the charging pad cluster. A similar confirmation procedure may be triggered by discovery of an absence of a previously known asset, or discovery of a displacement beyond a threshold distance of a previously known asset. Any indicated changes or modifications to the cluster that cannot be explained or verified as proper (e.g., via some form or planning or approval procedure) may then be subject to further review (e.g., by a human or automated review process) and/or may warrant physical inspection and/or maintenance at the cluster site.

700 711 Comparison between a new-generated map and a previously-existing map may further be used to produce an updated current reference map for a known charging pad cluster. This may involve resolving any discrepancies between the two maps of the comparison, and determining any updated map positions of previously known and tracked assets. The asset managermay then store the updated current reference map in reference mapsaccording to the cluster ID, and update asset records for assets in updated map by storing newly-received aerial observation data with the historical data, and updating map positions (e.g., Xm and Ym) in the records as may be necessary (e.g., as determined by the map comparison).

In accordance with example embodiments, map graph construction and reference map generation may include and/or involve additional operations besides those described above that are illustrative of the basic principles. Some of the additional operations may be considered ancillary to data analysis procedures involving observational data or measurements, for example. Others may relate to contingencies and/or special circumstances that may arise in a UAV system.

One additional operation may involve preprocessing of aerial observation data to detect and possibly discard observational outliers—e.g., observations or measurements that can be identified as erroneous or below some quality threshold. This operation is sometimes referred to colloquially as “data sanitation.” One example of outliers may include vector positions from poses having unreliable visibility to the associated assets, such as excessive ALG and/or horizontal distance from the cluster. Such vector positions may thus be unreliable. Another example may be measurements with uncertainties exceeding one or more thresholds.

After input data are sanitized, a correspondence between vectors and assets of the remaining data may be determined. Correspondence attempts to associate vectors with particular assets—e.g., charging pads and fiducials. This operation primarily applies when the (sanitized) vector data represent at least some multiply-observed assets. Correspondence of vectors with fiducial markers may be aided by identification markings on the fiducials, though, as described above, use of fiducial constellations allows non-unique markings within a charging pad cluster. However, by avoiding close proximity of identically marked fiducials during cluster deployment, fiducial markings may largely enable reliable determination of correspondence. Correspondence of vectors with charging pads may be more prone to errors, since charging pads are generally almost visibly indistinguishable. To the extent that mis-correspondences survive into the optimization process, result inaccuracies may be detected after the fact, and compensated by other techniques, such as iterative optimization passes with mis-correspondence corrections in between.

After a reference map is generated, further data sanitation may be carried out. This may involve detecting and removing assets that are misplaced in the map, as determined by some form of thresholding or heuristics. For example, an asset that appears beyond some threshold of the boundary of a charging pad cluster may be deemed errant and removed. As another example, if two distinct asset overlap in the map, one or both may be deemed errant. Other forms of post-map sanitation may be used as well.

In order for a UAV system to generate a flight mission plan for a UAV, and in order for the UAV to be able to fly a mission, the UAV's starting geolocation and orientation need to be known to some threshold degree of accuracy—e.g., 1-2 m. When a UAV first powers on, neither the UAV nor a UAV ground-support system in which the UAV operates will typically know its geolocation or orientation. Thus, a UAV that powers on needs its geolocation and orientation to be determined before a mission plan can be generated, and before the UAV can fly a mission. In addition, the UAV may need to perform various health and safety checks to achieve a state of flight readiness. These may include calibration of the UAV's navigation system according to the UAV's gcolocation and orientation.

While a UAV may determine its geolocation from GPS after powering on, the accuracy of such a determination may be limited due to direct or multipath interference of GPS signals at ground level where the UAV may be located when it powers on. One technique for improving GPS-based geolocation is for the UAV to ascend vertically to an AGL where GPS signal interference is reduced or eliminated. By way of example, this AGL could be 20 m. However, flights at AGLs of 20 m (and possibly somewhat lower) may be subject to aviation regulations, and/or incur various operational complexities related to mission planning for example. Thus, it would be desirable to be able to determine UAV initial geolocation and orientation after power-up without needing to rely on initial accurate GPS measurements (and correspondingly, without needing low-interference GPS signals), and/or without requiring flying above an AGL that might trigger aviation regulations or operational complexities.

Accordingly, the inventors have devised techniques and procedures for a UAV to use a low-altitude vertical flight to acquire visual data that can be transmitted to, and used by, a UAV ground-support system to determine the UAV's geolocation and orientation within a threshold accuracy. Because the UAV's geolocation and orientation may not be accurately determined when the visual data are acquired, the acquired visual data may be considered uncalibrated observations. Hence, the related systems and method described herein are referred to as initial UAV geolocation and orientation determination from uncalibrated positional observations. It should be understood that the techniques could also be applied to calibrated and/or accurate UAV positional observations as well.

Example embodiments may be understood in the context of charging pads and clusters, where UAVs typically start their missions after powering on. More particularly, a UAV may start a mission from a specific charging pad in a specific cluster. While the techniques and procedures described above for asset tracking and reference map generation may enable deployment of clusters without strict requirements for initial precise geolocations and orientations of charging pads and fiducials, repeated applications of map graph optimizations over numerous aerial observations from one or more UAVs may be expected to eventually yield reasonably accurate geolocations and orientations of tracked assets. And for clusters deployed in a mode that uses initial precise geolocations and orientation of cluster assets, corresponding reference maps may be accurate before—or even without—map graph optimization techniques. By way of example, geolocations of cluster assets in either mode of cluster deployment may be (or become) accurate to 1-2 m or less.

Thus, the identities (and thereby the known geolocations) of the specific cluster pad and specific cluster from which a UAV launches may be used to determine the UAV's initial geolocation. Similarly, known orientations of the particular pad and/or one or more nearby fiducials may be used to determine the UAV's orientation based on visual data, as described below. The concept of determining initial UAV geolocation and orientation from uncalibrated positional observations involves then, among other aspects, a UAV acquiring a low-altitude aerial image that sufficiently captures the UAV's launching pad directly below the UAV, as well as one or more nearby fiducials, for a server in a UAV support system to recognize and identify the assets in the image, and thereby determine their and the UAV's geolocation and orientation.

Example embodiments of initial UAV geolocation and orientation determination from uncalibrated positional observations may be further described by way of example operation. In an example usage scenario, a UAV in a powered-off state may be located on a particular pad in a particular cluster. The UAV may have placed on the particular pad by ground-support personnel in preparation for immediate or later power-on and mission activities, for example. In accordance with example embodiments, upon power-up, the UAV may determine its approximate geolocation and orientation (e.g., heading with respect to geographic or magnetic north) from GPS. As described above, the accuracy of the GPS-based geolocation obtained at ground level may be insufficient for mission planning or calibration of the UAV's navigation system. However, it may be sufficient to identify the charging cluster in which the UAV is located.

502 502 502 402 5 FIG. 5 FIG. In further accordance with example embodiments, the UAV may also undertake a low-altitude vertical ascent directly above its still-unidentified launching pad. In accordance with example embodiments, the UAV may ascend to only 2-5 m AGL or even as low as 1-2 m AGL. The exact height need not necessarily be specified exactly, other than configuring it to be below a height the might otherwise subject the flight to one or more aviation regulations and/or flight operation complexities, and high enough to obtain an image of the particular charging pad and at least one nearby fiducial. While hovering (or holding station) at the low altitude, the UAV may capture an image of the particular charging pad and a portion of the cluster surrounding the particular charging pad, and within which one or more fiducial markers are observed. An example of such an image could be similar to the reference constellations-A or-B shown in, for example, except that the field of view (FOV) from low altitude may encompass a smaller area than the ones depicted in. For example, a low-altitude image of reference constellation-A might include the charging pad-A and just the two or three closest fiducials.

For purposes of the present discussion, an image captured at low altitude as described above will be referred to herein as a “low-altitude image” in order to distinguish its intended purpose from other images that a UAV may capture, such as those in connection with context-based navigation described above. It should be understood, however, that the principles described below in connection with processing and analysis of a low-altitude image, and observational information contained therein, are not necessarily limited to images captured at low altitude.

300 700 310 100 700 700 7 FIG.A In accordance with example embodiments, the UAV may transmit both visual information in the image and its approximate geolocation and orientation to a server or other computing device or system in the UAV system, such as the asset managerin the central dispatch systemin, for example. The transmission may be carried via a wireless broadband cellular connection, such as that described above between UAVand the asset manager. The asset manager may identify the particular charging cluster, as well as a list of assets-pads and fiducials—in the vicinity of the UAV's reported approximate geolocation. The list of assets may be referred to as a “candidate list.” The candidate list and the approximate location of the UAV may thus provide a context for the asset managerto match against the received visual information, and thereby identify the particular charging pad with a specific charging pad of the cluster. The known location of the specific charging pad may then be taken as the UAV's geolocation at launch. The matching procedure may also be applied to the image information of the one or more fiducials and their respective orientations in the context of the list of assets and the UAV's reported approximate geolocation and orientation to derive an accurate determination of the UAV's orientation.

700 700 In further accordance with example embodiments, the visual information of the image transmitted by the UAV may be a processed and condensed form of the low-altitude image. In particular, the UAV may process the low-altitude image to generate a condensed version of visual information that is descriptive of the particular charging pad and one or more fiducial markers captured in the image. A condensed data format may enable transmission of the information to the asset manager, via a wireless broadband cellular connection, using minimal bandwidth to incur low cost and low delay, particularly compared with otherwise transmitting a full digital image. By way of example, condensed information could specify pixel locations in the image of the particular charging pad and the one or more fiducials, as well as a respective identifying marking on each fiducial and an orientation indicator on each. This information could thus be sufficient for the asset managerto reconstruct essential outlines of the image—e.g., relative locations and orientations of the particular pad and the one more fiducials-without having or needing the full digital low-altitude image. Other forms of image processing by the UAV and/or other formats of condensed image information may be used in addition or instead of the example above.

700 In accordance with example embodiments, the UAV may use its approximate geolocation from which it acquired the low-altitude image, together with the condensed image information derived from the UAV processing the acquired image, to construct a form of low-altitude observation data for its transmission to the asset manager. For purposes of the present discussion, and in consideration of the low-altitude image typically capturing just a portion of the cluster, the low-altitude observation data will be referred to herein as “cluster-portion observation data.” This term may also help distinguish from the term “aerial observation data” used above. Cluster-portion observation data may include other information, such as an identity of the UAV that generated and transmitted the observation data.

700 Upon receiving cluster-portion observation data from a UAV, an asset manager(or other server) may reconstruct essential outlines of the image, as described above. In an example, a reconstructed image outline may be thought of conceptually as just boundary outlines of a charging pad and one or more nearby fiducial captured in the corresponding low-altitude image. The reconstructed outline fiducials may also include identifying markings and orientation indicators. Additionally or alternatively, identifying markings and orientation indicators of the one or more fiducials may be determined by the UAV during processing, and included as data items in the cluster-portion observation data. Thus, while a charging pad-particularly one represented only in outline form, for example—may be indistinguishable from any other charging pad in the candidate list (or possibly anywhere in the cluster), the identifying markings on the outlined fiducials may allow the asset manager to identify specific fiducials in the candidate list.

700 700 700 Once the one or more outline fiducials in the reconstructed outline image are matched with one or more specific fiducials in the candidate list, the orientation indicators on (or associated with) the outline fiducials may be used to align the reconstructed image with known orientations of the specific fiducials. In an example, alignment could correspond to a rotation. With the reconstructed image oriented according to the specific fiducials, the asset managermay then match the particular charging pad in the reconstructed image with a known charging pad in the candidate list. A successful match thus identifies the particular charging pad from which the UAV launched its low-altitude flight. Further, the reorientation of the reconstructed image may be used to accurately determine the UAV's orientation. The asset managermay thus determine the UAV's geolocation and orientation for mission planning, and for reporting to the UAV. The UAV may then receive its accurate geolocation and orientation from the asset manager(or other server), and thereby apply any necessary and/or appropriate calibration of its navigation system.

It may be noted that in both context-based navigation and asset tracking and management, described by way of example above, unique identifications of fiducials in a cluster need not necessarily be strictly enforced. However, with reasonable, minimal effort, the distribution of fiducials in a given cluster may be arranged such that any two or more fiducials with the same identification are spaced far enough apart to avoid confusing them with each other. Thus, the ability to match outlined assets in a reconstructed image with assets in a candidate list, while not necessarily guaranteed in every instance or application of the described operations, may be reasonable expected to be successful in most or (possibly all) instances.

700 700 In accordance with example embodiments, during a UAV's low-altitude flight above a not-yet-identified launch pad, it may obtain multiple low-altitude images and generate multiple cluster-portion observation data transmissions to the asset manager. The asset managermay, in turn, carry out multiple image reconstruction and asset matching operations in order to help enhance reliability of, and/or confidence in, identification of the UAV's launching pad. In an example, if N separate determinations of charging pad identification are carried out, as described above, it may be specified that a threshold number of the N need to yield the same identification in order achieve at least a minimum confidence level of the identification. For example, a minimum confidence level could correspond to 0.8×N (e.g., converted to an integer minimum). Other fractional values could be used.

700 700 700 700 Notwithstanding an expected high degree of reliable launch pad identification using uncalibrated positional observations from a low-altitude UAV flight, example embodiments further include operations for confirming launch pad identification prior to a UAV departing its launch site for a mission flight. Thus, in accordance with example embodiments, a UAV may carry out further confirming operations after receiving its geolocation and orientation from the asset manager(or other server), receiving a mission flight plan, and calibrating its navigation system. More particularly, when a UAV departs on a mission, it may first ascend to approximately 20 m directly above its launching pad. At this AGL, the UAV may obtain a GPS-based geolocation that is sufficiently accurate to check against the geolocation determined by the asset manager. If the UAV finds that it GPS-based geolocation agrees with the server-determined geolocation to a specified threshold, then the UAV may proceed on its mission. If, instead, the UAV discovers a disagreement greater than the threshold, it may postpose or abort the mission, and report the GPS-based geolocation to the asset manager(or other server). The asset managermay then reformulate the UAV's mission flight plan according to the GPS-based geolocation, and resend the update mission plan to the UAV. The UAV may then recalibrate its navigation system and proceed on the mission according to the updated plan.

700 700 In the event of reformulation of the plan based on the GPS location, the UAV may remain hovering at 20 m (or thereabouts) while awaiting the updated from the asset manager. Alternatively, the UAV may return to its launching pad while awaiting the updated plan. In still another alternative or option of operation, if a UAV discovers that its server-based geolocation disagrees with the GPS-based geolocation to greater than the threshold, the UAV and the asset managermay try again to determine the UAV's geolocation and orientation from one or more additional low-altitude images. While this alternative may not be necessary since the UAV will have already obtained a GPS-based geolocation, there may be circumstances in which it is a viable and/or beneficial approach. In any case, this alternative need not be ruled out by the availability of a GPS-based geolocation determination.

In further accordance with example embodiments, a flight plan may include, among other information, one or more waypoints between its launching pad and landing pad of a mission. A waypoint is a predefined geolocation against which a UAV may check its navigation-based location to ensure it is following a flight plan correctly. One technique for implementing a check of a server-based geolocation against a GPS-based geolocation is to include the 20 m checkpoint above the launch pad as the first waypoint in the flight plan. Thus, the UAV may follow routine flight plan operations to its first waypoint, where it may then check the GPS-based geolocation against its server-based geolocation. Other techniques for implementing the check are possible as well.

10 FIG.A 10 FIG.A 1000 1000 1000 1002 1004 1006 1008 4010 1000 1012 is a simplified block diagram of an example computing system (or computing device). The computing systemcan be configured to perform and/or can perform one or more acts, such as the acts described in this disclosure. As shown, the computing devicemay include processor(s), memory, network interface(s), and an input/output unit. By way of example, the components are communicatively connected by a bus. The bus could also provide power from a power supply (not shown). Computing systemmay also include a database, represented as an external component in the example of.

1002 1002 1004 Processorsmay include one or more general purpose processors and/or one or more special purpose processors (e.g., digital signal processors (DSPs) or graphics processing units (GPUs). Processorsmay be configured to execute computer-readable instructions that are contained in memoryand/or other instructions as described herein.

1004 404 1002 1004 1004 1004 Memorymay include firmware, a kernel, and applications, among other forms and functions of memory. As described, the memorymay store machine-language instructions, such as programming code or non-transitory computer-readable storage media, which may be executed by the processorin order to carry out operations that implement the methods, scenarios, and techniques as described herein. In some examples, memorymay be implemented using a single physical device (e.g., one magnetic or disc storage unit), while in other examples, memorymay be implemented using two or more physical devices. In some examples, memorymay include storage for one or more machine learning systems and/or one or more machine learning models as described herein.

1000 1006 1008 1004 In some instances, the computing systemcan execute program instructions in response to receiving an input, such as an input received via the network interfaceand/or the user interface. The memorycan also store other data, such as any of the data described in this disclosure.

1006 1000 1006 406 The network interfacecan allow the computing systemto connect with and/or communicate with another entity according to one or more protocols. In one example, the communication interfacecan be a wired interface, such as an Ethernet interface. In another example, the communication interfacecan be a wireless interface, such as a cellular or WiFi interface.

1008 1000 1000 1008 The user interfacecan allow for interaction between the computing systemand a user of the computing system, if applicable. As such, the user interfacecan include, or provide an interface connection to, input components such as a keyboard, a mouse, a touch-sensitive panel, and/or a microphone, and/or output components such as a display device (which, for example, can be combined with a touch-sensitive panel), and/or a sound speaker.

1000 1000 1000 1010 1000 10 FIG.A The computing systemcan also include one or more connection mechanisms that connect various components within the computing system. For example, the computing systemcan include a connection mechanismthat connects components of the computing system, as shown in.

1006 1000 1000 Network interface(s)may provide network connectivity to the computing system, such as to the internet or other public and/or private networks. Networks may be used to connect the computing systemwith one or more other computing devices, such as servers or other computing systems. In an example embodiment, multiple computing systems could be communicatively connected, and example methods could be implemented in a distributed fashion.

1012 Databasemay include storage for input and/or output data, such as records for tracking cluster assets.

1000 1000 700 3 FIG. In some configurations, the computing systemcan include one or more of the above-described components and can be arranged in various ways. For example, the computer systemcan be configured as a server, such as the asset manager, or other components depicted in, for example.

10 FIG.B 10 FIG.B 1024 1000 1026 1028 1030 1034 1026 1028 1030 1024 1024 depicts a cloud-based server clusterin accordance with example embodiments. In, operations of a computing device (e.g., computing device) may be distributed between server devices, data storage, and routers, all of which may be connected by local cluster network. The number of server devices, data storages, and routersin server clustermay depend on the computing task(s) and/or applications assigned to server cluster.

1026 1000 1026 1024 1026 For example, server devicescan be configured to perform various computing tasks of computing device. Thus, computing tasks can be distributed among one or more of server devices. To the extent that these computing tasks can be performed in parallel, such a distribution of tasks may reduce the total time to complete these tasks and return a result. For purposes of simplicity, both server clusterand individual server devicesmay be referred to as a “server device.” This nomenclature should be understood to imply that one or more distinct server devices, data storage devices, and cluster routers may be involved in server device operations.

1032 1032 1026 1032 1026 1032 Data storage, also referred to database, may be data storage arrays that include array controllers configured to manage read and write access to groups of hard disk drives and/or solid state drives. The drive array controllers, alone or in conjunction with server devices, may also be configured to manage backup or redundant copies of the data stored in databaseto protect against drive failures or other types of failures that prevent one or more of server devicesfrom accessing units of data storage. Other types of memory aside from drives may be used.

1030 1024 1030 1026 1032 1032 1024 1022 1020 Routersmay include networking equipment configured to provide internal and external communications for server cluster. For example, routersmay include one or more packet-switching and/or routing devices (including switches and/or gateways) configured to provide (i) network communications between server devicesand data storagevia local cluster network, and/or (ii) network communications between server clusterand other devices via communication linkto network.

1030 1026 1032 1034 1022 Additionally, the configuration of routerscan be based at least in part on the data communication requirements of server devicesand data storage, the latency and throughput of the local cluster network, the latency, throughput, and cost of communication link, and/or other factors that may contribute to the cost, speed, fault-tolerance, resiliency, efficiency, and/or other design goals of the system architecture.

1032 1032 As a possible example, data storagemay include any form of database, such as a structured query language (SQL) database. Various types of data structures may store the information in such a database, including but not limited to tables, arrays, lists, trees, and tuples. Furthermore, any databases in data storagemay be monolithic or distributed across multiple physical devices.

1026 1032 1026 1026 Server devicesmay be configured to transmit data to and receive data from data storage. This transmission and retrieval may take the form of SQL queries or other types of database queries, and the output of such queries, respectively. Additional text, images, video, and/or audio may be included as well. Furthermore, server devicesmay organize the received data into web page or web application representations. Such a representation may take the form of a markup language, such as the hypertext markup language (HTML), the extensible markup language (XML), or some other standardized or proprietary format. Moreover, server devicesmay have the capability of executing various types of computerized scripting languages, such as but not limited to Perl, Python, PHP Hypertext Preprocessor (PHP), Active Server Pages (ASP), JAVASCRIPT®, and so on. Computer program code written in these languages may facilitate the providing of web pages to client devices, as well as client device interaction with the web pages. Alternatively or additionally, JAVA® may be used to facilitate generation of web pages and/or to provide web application functionality.

11 FIG. 7 FIG. 1100 1100 1100 700 700 1000 1024 is a block diagram of method, in accordance with example embodiments. More particularly, example methodillustrates an example embodiment of automated discovery and monitoring of UAV ground-support infrastructure. In some examples, methodmay be carried out by a server in an infrastructure support network for UAVs, such as the asset managerrepresented in. As described, the asset manager, or other server, may be implemented in or by a computing system, such as computing systemand/or server cluster, for example. Example methods may be implemented as computer-readable instructions that, when executed by one or more processors of a server or computing system, cause the server or computing system to carry out various operations of the method. Further, the computer-readable instructions may be stored in memory of the server or computing system that is accessible to the one or more processors. The computer-readable instructions may also be stored on one or more forms of storage media, including non-transitory computer-readable media. In addition to serving as a tangible product of manufacture for storage of computer-readable instructions, such storage media may also be used for transporting the instructions to a computing system in order to transfer or load the instructions into memory of the computing system.

1102 1100 At block, methodincludes receiving first aerial observation data of a particular ground-based cluster of charging pads for UAVs from a first UAV. The particular cluster comprises cluster assets may include the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout. The first aerial observation data may include position measurements of the first UAV at each of one or more aerial geolocations above a locale of the particular cluster, and vector positions of at least a first subset of the cluster assets with respect to the one or more aerial geolocations.

1104 1100 At block, methodincludes generating a map graph from at least the first aerial observation data. The map graph may include (i) nodes corresponding to both the one or more aerial geolocations and the vector positions of the at least first subset of the cluster assets, and (ii) edges between pairs of selected nodes, the edges corresponding to distances between selected nodes and including measurement uncertainties.

1106 1100 Finally, at block, methodincludes generating a particular spatial map of at least a particular subset of cluster assets of the particular ground-based cluster by computationally optimizing the map graph.

1100 In accordance with example embodiments, methodmay further include receiving at least second aerial observation data of the particular ground-based cluster from at least a second UAV. The at least second aerial observation data may include position measurements of the at least second UAV at each of one or more second aerial geolocations above the locale of the particular cluster, and vector positions of at least a second subset of the cluster assets with respect to the one or more second aerial geolocations. In this case, generating the map graph from at least the first aerial observation data may include generating the map graph from the first aerial observation data and the at least second aerial observation data. Also in this case, the map graph may further include (i) nodes corresponding to both the one or more second aerial geolocations and the vector positions of the at least second subset of the cluster assets, and (ii) edges between pairs of selected nodes.

In further accordance with example embodiments, the first subset of cluster assets and the second subset of cluster assets may include at least partially overlapping subsets.

In accordance with example embodiments, receiving the first aerial observation data may entail receiving the first aerial observation data during a first time interval, and receiving the at least second aerial observation data of the particular ground-based cluster may entail receiving at least a portion of the at least second aerial observation data during a second time interval that is concurrent with the first time interval.

1100 In accordance with example embodiments, receiving the first aerial observation data may entail receiving the first aerial observation data during a first time interval, and receiving the at least second aerial observation data of the particular ground-based cluster may entail receiving at least a portion of the at least second aerial observation data during a second time interval prior to the first time interval. In this case, methodmay further include: generating a prior spatial map of prior assets of the particular ground-based cluster based on an optimized map graph generated from the at least a portion of the at least second aerial observation data; and in a database associated with the computing system, storing (i) records of the prior assets of particular ground-based cluster, each record tracking a respective prior cluster asset according to information including an identifier, an optimally-derived position of the respective prior cluster asset, and prior vector locations of the respective prior cluster asset from the at least a portion of the at least second aerial observation data, and (ii) the prior spatial map.

In accordance with example embodiments, the first subset of the cluster assets may include all of the cluster assets of the particular ground-based cluster, and the particular subset of cluster assets of the particular cluster all of the cluster assets in the particular spatial map, or fewer than all of the cluster assets in the particular spatial map.

712 1100 In accordance with example embodiments, the computing system may include or be associated with a database, such as asset database. The database may include records of known cluster assets of known ground-based clusters at known locales, a known spatial map of each known ground-based cluster. Each record may track a respective known cluster asset according to information including an identifier, an optimally-derived position of the respective known cluster asset, and previous vector locations of the respective known cluster asset from previously received aerial observation data from one or more UAVs. In this arrangement, methodmay further include determining that the particular cluster ground-based cluster and its particular subset of cluster assets are not among the known ground-based clusters and/or the known cluster assets, and updating the database. Updating the database may involve adding the particular ground-based cluster and its particular subset of assets in the records of known cluster assets, and adding the particular spatial map to the known spatial maps.

11 In accordance with example embodiments, methodmay further include providing the particular spatial map to one or more UAVs that flight missions to the particular ground-based cluster.

712 1100 In accordance with example embodiments, the computing system may include or be associated with a database, such as asset database. The database may include records of known cluster assets of known ground-based clusters at known locales, and a known spatial map of each known ground-based cluster. Each record may track a respective known cluster asset according to information including an identifier, an optimally-derived position of the respective known cluster asset, and previous vector locations of the respective known cluster asset from previously received aerial observation data from one or more UAVs. In this arrangement, methodmay further include determining that the particular ground-based cluster and its particular subset of cluster assets are among the known ground-based clusters and/or the known cluster assets; updating database records tracking the known cluster assets of the particular ground-based cluster with updated optimally-derived positions of the known cluster assets and with the vector locations of the known cluster assets from the received first aerial observation data; and updating the known spatial map of the particular ground-based cluster with the generated particular spatial map. Further, generating the map graph from at least the first aerial observation data may involve generating the map graph from the first aerial observation data and the previously received aerial observation data of the particular ground-based cluster. The map graph may further include (i) nodes corresponding to both one or more historical aerial geolocations and historical vector positions of previously identified cluster assets from the previously received aerial observation data, and (ii) edges between pairs of selected nodes.

In accordance with example embodiments, updating the database records tracking the known cluster assets of the particular ground-based cluster with updated optimally-derived positions of the known cluster assets may involve computationally optimizing the map graph comprising the first aerial observation data and the previously-received aerial observation data to generate the particular spatial map, and determining the updated optimally-derived positions according to positions of assets identified in the updated known spatial map.

In accordance with example embodiments, updating the known spatial map of the particular ground-based cluster with the generated particular spatial map may involve computationally optimizing the map graph that includes the first aerial observation data and the previously-received aerial observation data to generate the particular spatial map, and making a comparison of the particular spatial map with the known spatial map. Updating the known may then also include adding, removing, and/or updating assets and their positions in the known spatial map based on the comparison.

1100 In accordance with example embodiments, methodmay further include providing the updated known spatial map to one or more UAVs that flight missions to the particular ground-based cluster.

1100 In some examples, a non-transitory computer readable medium may include program instructions executable by one or more processors to perform operations. The operations may include the steps of method.

1100 700 Example methodpresents automated discovery and monitoring of UAV ground-support infrastructure, by way of example, in terms of operations of a backend server, such as the asset manageror other computing system, in a UAV ground-support system. Example methods may also be specified in terms of a system that includes a server, a charging pad cluster, and at least one UAV. More specifically, a system may include a server in an infrastructure support network UAVs, a ground-based cluster of charging pads for UAVs, the ground-based cluster comprising cluster assets including the charging pads arranged in a layout and a plurality of fiducial markers distributed at positions across the layout, and a first a first UAV comprising a navigation system, a first imaging system, and a first control system.

1100 As with example method, methods specified in terms of multiple participating (or coordinated) components of a system may be implemented as computer-readable instructions that, when executed by one or more processors of the system, cause the system to carry out various operations of the method. The computer-readable instructions may be stored in memory of the one or more components of the system that are accessible to the one or more processors. The computer-readable instructions may also be stored on one or more forms of storage media, including non-transitory computer-readable media. An example of a method specified in terms of a server UAVs may be described as follows.

In accordance with example embodiments, a first control system of a first UAV may be configured to cause the first UAV to fly to one or more first aerial geolocations above the ground-based cluster, identify each of one or more first assets of the ground-based cluster from one or more aerial images captured by the first imaging system at the one or more first aerial geolocations, and determine first vector positions of the one or more first assets with respect to the one or more first aerial geolocations. The first control system may then send first aerial observation data to the server. The first aerial observation data may include position measurements of the first UAV at each of the one or more first aerial geolocations, and the first vector positions of the one or more first assets.

Also in accordance with example embodiments, the server may be configured to receive the first aerial observation data, and generate a map graph from at least the first aerial observation data. The map graph may include (i) nodes corresponding to both the one or more first aerial geolocations and the first vector positions of the one or more first cluster assets, and (ii) edges between pairs of selected nodes, the edges corresponding to distances between selected nodes and including measurement uncertainties. The server may then generate a spatial map of at least the one or more first cluster assets of the ground-based cluster by computationally optimizing the map graph.

In accordance with example embodiments, the origin of the flight of the first UAV to the one or more first aerial geolocations may be of a ground-based position within the ground-based cluster, or a ground-based position from within a different ground-based cluster.

In accordance with example embodiments, the first aerial observation data may further include orientation measurements of the first UAV at each of the one or more first aerial geolocations, and the position measurements of the first UAV at each of the one or more first aerial geolocations may be determined by a GPS system.

In accordance with example embodiments, the system may include a second UAV having a navigation system, a second imaging system, and a second control system. The second control system may be configured to cause the second UAV to fly to one or more second aerial geolocations above the ground-based cluster; identify each of one or more second assets of the ground-based cluster from one or more aerial images captured by the second imaging system at the one or more second aerial geolocations; determine second vector positions of the one or more second assets with respect to the one or more second aerial geolocations; and send second aerial observation data to the server. The second aerial observation data may include position measurements of the second UAV at each of the one or more second aerial geolocations, and the second vector positions of the one or more second assets. In this arrangement, the server may be further configured to receive the second aerial observation data. Further, generating the map graph from at least the first aerial observation data may involve generating the map graph from the first aerial observation data and the second aerial observation data. The map graph may further include (i) nodes corresponding to both the one or more second aerial geolocations and the second vector positions of the one or more second assets, and (ii) edges between pairs of selected nodes. Also in this arrangement, generating the spatial map of the at least the one or more first assets of the ground-based cluster may involve generating the spatial map of the one or more first and second assets of the ground-based cluster by computationally optimizing the map graph, and accounting for any overlapping first and second assets.

The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims.

The above-detailed description describes various features and functions of the disclosed systems, devices, and methods with reference to the accompanying figures. In the figures, similar symbols typically identify similar components unless context dictates otherwise. The example embodiments described herein and in the figures are not meant to be limiting. Other embodiments can be utilized, and other changes can be made without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.

A block that represents a processing of information may correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a block that represents a processing of information may correspond to a module, a segment, or a portion of program code (including related data). The program code may include one or more instructions executable by a processor for implementing specific logical functions or actions in the method or technique. The program code or related data may be stored on any type of computer-readable medium such as a storage device including a disk or hard drive or other storage medium.

The computer-readable medium may also include non-transitory computer-readable media such as computer-readable media that stores data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer-readable media may also include non-transitory computer-readable media that stores program code or data for longer periods of time, such as secondary or persistent long-term storage, like read-only memory (ROM), optical or magnetic disks, compact-disc read-only memory (CD-ROM), for example. The computer-readable media may also be any other volatile or non-volatile storage systems. A computer-readable medium may be considered a computer-readable storage medium, for example, or a tangible storage device.

Moreover, a block that represents one or more information transmissions may correspond to information transmissions between software or hardware modules in the same physical device. However, other information transmissions may be between software modules or hardware modules in different physical devices.

The particular arrangements shown in the figures should not be viewed as limiting. It should be understood that other embodiments can include more or less of each element shown in a given figure. Further, some of the illustrated elements can be combined or omitted. Yet further, an example embodiment can include elements that are not illustrated in the figures.

While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.