Systems, apparatuses, methods, and computer program products for optimizing search latency

Embodiments of the present disclosure relate generally to optimizing search latency.

Applicant has identified many technical challenges and difficulties associated with optimizing search latency. Through applied effort, ingenuity, and innovation, Applicant has solved problems related to optimizing search latency by developing solutions embodied in the present disclosure, which are described in detail below.

Various embodiments described herein relate to optimizing search latency.

In accordance with one aspect of the disclosure, a computer-implemented method is provided. In some embodiments, the computer-implemented method may include receiving one or more datasets from a user. In some embodiments, the computer-implemented method may include storing at least a portion of the one or more datasets in a first shard during a first time period. In some embodiments, the first shard includes shard data. In some embodiments, the shard data is associated with a data size. In some embodiments, the computer-implemented method may include determining that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, the computer-implemented method may include, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, transferring at least a portion of the shard data in the first shard to a second shard.

In some embodiments, transferring at least a portion of the shard data in the first shard to the second shard occurs during a third time period.

In some embodiments, the third time period is based at least in part on historical activity data associated with the user.

In some embodiments, the historical activity data indicates that the user is inactive during the third time period.

In some embodiments, the computer-implemented method may include receiving a first query from the user to access at least a portion of the shard data during a second time period.

In some embodiments, the first query is associated with a first latency.

In some embodiments, the computer-implemented method may include receiving a second query from the user to access at least a portion of the shard data during a fourth time period.

In some embodiments, the second query is associated with a second latency.

In some embodiments, the first latency is greater than the second latency.

In some embodiments, the shard data comprises at least a portion of the one or more datasets.

In some embodiments, the first shard is associated with a first database and the second shard is associated with a second database.

In some embodiments, the first shard has a first data storage size.

In some embodiments, the first data storage size is greater than the first data size threshold.

In some embodiments, the computer-implemented method may include determining that a second data size of second shard data in a third shard is less than a second data size threshold.

In some embodiments, the computer-implemented method may include in response to the determination that the second data size of the second shard data in the third shard is less than the second data size threshold, transferring at least a portion of the second shard data in the third shard to a fourth shard.

In accordance with another aspect of the disclosure, an apparatus is provided. In some embodiments, the apparatus may include at least one processor and at least one memory coupled to the at least one processor. In some embodiments, the at least one processor is configured to receive one or more datasets from a user. In some embodiments, the at least one processor is configured to store at least a portion of the one or more datasets in a first shard during a first time period. In some embodiments, the first shard includes shard data. In some embodiments, the shard data is associated with a data size. In some embodiments, the at least one processor is configured to determine that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, the at least one processor is configured to, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, transfer at least a portion of the shard data in the first shard to a second shard.

In some embodiments, transferring at least a portion of the shard data in the first shard to the second shard occurs during a third time period.

In some embodiments, the third time period is based at least in part on historical activity data associated with the user.

In some embodiments, the historical activity data indicates that the user is inactive during the third time period.

In some embodiments, the at least one processor is configured to receive a first query from the user to access at least a portion of the shard data during a second time period.

In some embodiments, the first query is associated with a first latency.

In some embodiments, the at least one processor is configured to receive a second query from the user to access at least a portion of the shard data during a fourth time period.

In some embodiments, the second query is associated with a second latency.

In some embodiments, the first latency is greater than the second latency.

In some embodiments, the shard data comprises at least a portion of the one or more datasets.

In some embodiments, the first shard is associated with a first database and the second shard is associated with a second database.

In some embodiments, the first shard has a first data storage size.

In some embodiments, the first data storage size is greater than the first data size threshold.

In some embodiments, the at least one processor is configured to determine that a second data size of second shard data in a third shard is less than a second data size threshold.

In some embodiments, the at least one processor is configured to, in response to the determination that the second data size of the second shard data in the third shard is less than the second data size threshold, transfer at least a portion of the second shard data in the third shard to a fourth shard.

In accordance with another aspect of the disclosure, a non-transitory computer-readable storage medium is provided. In some embodiments, the non-transitory computer-readable storage medium may include computer program code for execution by one or more processors of a device. In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to receive one or more datasets from a user. In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to store at least a portion of the one or more datasets in a first shard during a first time period. In some embodiments, the first shard includes shard data. In some embodiments, the shard data is associated with a data size. In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to determine that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, transfer at least a portion of the shard data in the first shard to a second shard.

In some embodiments, transferring at least a portion of the shard data in the first shard to the second shard occurs during a third time period.

In some embodiments, the third time period is based at least in part on historical activity data associated with the user.

In some embodiments, the historical activity data indicates that the user is inactive during the third time period.

In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to receive a first query from the user to access at least a portion of the shard data during a second time period.

In some embodiments, the first query is associated with a first latency.

In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to receive a second query from the user to access at least a portion of the shard data during a fourth time period.

In some embodiments, the second query is associated with a second latency.

In some embodiments, the first latency is greater than the second latency.

In some embodiments, the shard data comprises at least a portion of the one or more datasets.

In some embodiments, the first shard is associated with a first database and the second shard is associated with a second database.

In some embodiments, the first shard has a first data storage size.

In some embodiments, the first data storage size is greater than the first data size threshold.

In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to determine that a second data size of second shard data in a third shard is less than a second data size threshold.

In some embodiments, the computer program code is configured to, when executed by the one or more processors, cause the device to, in response to the determination that the second data size of the second shard data in the third shard is less than the second data size threshold, transfer at least a portion of the second shard data in the third shard to a fourth shard.

The above summary is provided merely for purposes of summarizing some example embodiments to provide a basic understanding of some aspects of the present disclosure. Accordingly, it will be appreciated that the above-described embodiments are merely examples and should not be construed to narrow the scope or spirit of the disclosure in any way. It will be appreciated that the scope of the present disclosure encompasses many potential embodiments in addition to those here summarized, some of which will be further described below.

Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of disclosure are shown. Indeed, embodiments of the disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

Example embodiments disclosed herein address technical problems associated with optimizing search latency. As would be understood by one skilled in the field to which this disclosure pertains, there are numerous example scenarios in which optimizing search latency is desirable.

In many applications, it is necessary to store data in one or more databases, such that the data may be queried and, in response to a query, search results may be returned having data relevant to the query. For example, it may be necessary to store data about documents associated with a document management system in one or more databases such that the documents may be queried. In some such applications, the one or more databases may include a sharded database in which the data may be stored in one or more of a plurality of shards.

Example solutions for storing data in one or more sharded databases include, for example, a system configured to receive data, store the data in a first shard of the plurality of shards until the first shard is full, and successively store data in each of the plurality of shards until each shard in the plurality of shards is full (e.g., a data size associated with data stored in each shard is equal to a data storage size of each shard) and/or all of the data has been stored in one or more of the plurality of shards. However, such an example solution for storing data in one or more sharded databases is associated with high latency in response to queries of the data (e.g., it takes a large amount of time to return search results having data relevant to the query) because it takes the system longer to return search results to a query when the data size of the data in a shard is large and/or when the data size of the data in a shard is close to the data storage size of the shard (e.g., when the shard is greater than 50% full).

Thus, to address these and/or other issues related to optimizing search latency, example systems, apparatuses, computer program products, and/or methods are disclosed herein. For example, in an embodiment in this disclosure, described in greater detail below, a computer-implemented method may include receiving one or more datasets from a user. In some embodiments, the computer-implemented method may include storing at least a portion of the one or more datasets in a first shard during a first time period. In some embodiments, the first shard comprises shard data. In some embodiments, the shard data is associated with a data size. In some embodiments, the computer-implemented method may include determining that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, transferring at least a portion of the shard data in the first shard to a second shard. Accordingly, the systems, apparatuses, methods, and computer program products disclosed herein are able to optimize searching latency by monitoring the amount of shard data stored in each of a plurality of shards and transferring shard data to another shard when the data size of shard data in one shard is greater than a data size threshold to reduce latency associated with queries of shard data stored in one of the plurality of shards.

Embodiments of the present disclosure herein include systems, apparatuses, methods, and computer program products related to optimizing search latency. It should be readily appreciated that the embodiments of the apparatus, systems, methods, and computer program product described herein may be configured in various additional and alternative manners in addition to those expressly described herein.

1 FIG. 1 FIG. 100 110 110 110 110 110 illustrates an exemplary block diagram of an environmentin which embodiments of the present disclosure may operate. Specifically,illustrates a plurality of computing devices. In some embodiments, each of the plurality of computing devicesmay be associated with a user. In some embodiments a user may be an owner and/or operator of an associated computing device. In some embodiments, each computing device of the plurality of computing devicesis configured via hardware, software, firmware, and/or a combination thereof, to perform data intake of one or more types of data associated with an associated user. Additionally or alternatively still, in some embodiments, each computing device of the plurality of computing devicesis configured via hardware, software, firmware, and/or a combination thereof, to perform data reporting and/or other data output process(es) associated with one or more documents associated with an associated user. In some embodiments, each computing device of the plurality of computing devicesmay comprise a document management system.

110 110 110 In some embodiments, each of the plurality of computing devicesis associated with a determinable location. The determinable location of each of the plurality of computing devicesin some embodiments represents an absolute position (e.g., GPS coordinates, latitude, and longitude locations, and/or the like) or a relative position (e.g., an identifier representing the of each of the plurality of computing devicesfrom a local origin point).

130 130 130 130 130 100 130 The networkmay be embodied in any of a myriad of network configurations. In some embodiments, the networkmay be a public network (e.g., the Internet). In some embodiments, the networkmay be a private network (e.g., an internal localized, or closed-off network between particular devices). In some other embodiments, the networkmay be a hybrid network (e.g., a network enabling internal communications between particular connected devices and external communications with other devices). In various embodiments, the networkmay include one or more base station(s), relay(s), router(s), switch(es), cell tower(s), communications cable(s), routing station(s), and/or the like. In various embodiments, components of the environmentmay be communicatively coupled to transmit data to and/or receive data from one another over the network. Such configuration(s) include, without limitation, a wired or wireless Personal Area Network (PAN), Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), and/or the like.

100 140 140 110 160 150 140 110 140 110 140 150 110 140 150 110 110 140 In some embodiments, the environmentmay include a searching optimization system. The searching optimization systemmay be electronically and/or communicatively coupled to the plurality of computing devices, the one or more user devices, and/or the one or more databases. The searching optimization systemmay be located remotely, in proximity of, and/or within a particular computing device of the plurality of computing devices. In some embodiments, the searching optimization systemis configured via hardware, software, firmware, and/or a combination thereof, to perform data intake of one or more types of data associated with one or more of the plurality of computing devices. Additionally or alternatively, in some embodiments, the searching optimization systemis configured via hardware, software, firmware, and/or a combination thereof, to generate and/or transmit command(s) that control, adjust, or otherwise impact operations of one or more of, the one or more databasesand/or the plurality of computing devices. Additionally or alternatively still, in some embodiments, the searching optimization systemis configured via hardware, software, firmware, and/or a combination thereof, to perform data reporting and/or other data output process(es) associated with monitoring or otherwise analyzing operations of one or more of the one or more databasesand/or the plurality of computing devices, for example for generating and/or outputting report(s) corresponding to the operations performed via the plurality of computing devices. For example, in various embodiments, the searching optimization systemmay be configured to execute and/or perform one or more operations and/or functions described herein.

150 150 110 150 140 150 140 140 150 140 140 150 140 160 150 The one or more databasesmay be configured to receive, store, and/or transmit data. In some embodiments, the one or more databasesmay be associated with one or more datasets associated with the plurality of computing devices. In some embodiments, the one or more databasesmay be associated with datasets received by the searching optimization systemin real-time. Additionally or alternatively, the one or more databasesmay be associated with datasets received by the searching optimization systemon a periodic basis (e.g., the datasets may be received by the searching optimization systemonce per day). Additionally or alternatively, the one or more databasesmay be associated with datasets received by the searching optimization systemafter the searching optimization systemhas requested the datasets. Additionally or alternatively, the one or more databasesmay be associated with datasets based on an input (e.g., a user input) into the searching optimization systemand/or the one or more user devices. In some embodiments, the one or more databasesmay be sharded databases.

160 140 140 160 160 140 160 140 The one or more user devicesmay be associated with users of searching optimization system. In various embodiments, the searching optimization systemmay generate and/or transmit a message, alert, or indication to a user via one or more user devices. Additionally, or alternatively, the one or more user devicesmay be utilized by a user to remotely access a searching optimization system. This may be by, for example, an application operating on the one or more user devices. A user may access the searching optimization systemremotely, including one or more visualizations, reports, and/or real-time displays.

1 FIG. 130 140 150 110 Additionally, whileillustrates certain components as separate, standalone entities communicating over the network, various embodiments are not limited to this configuration. In other embodiments, one or more components may be directly connected and/or share hardware or the like. For example, in some embodiments, the searching optimization systemmay include one or more databases, which may collectively be located in or at the plurality of computing devices.

2 FIG. 2 FIG. 200 200 200 200 140 110 160 150 200 202 204 206 208 210 200 illustrates an exemplary block diagram of an example apparatus that may be specially configured in accordance with an example embodiment of the present disclosure. Specifically,depicts an example computing apparatus(“apparatus”) specially configured in accordance with at least some example embodiments of the present disclosure. For example, the computing apparatusmay be embodied as one or more of a specifically configured personal computing apparatus, a specifically configured cloud based computing apparatus, and/or the like. Examples of an apparatusmay include, but is not limited to, the searching optimization system, the plurality of computing devices, the one or more user devices, and/or the one or more databases. The apparatusincludes processor, memory, input/output circuitry, communications circuitry, and/or optional artificial intelligence (“AI”) and machine learning circuitry. In some embodiments, the apparatusis configured to execute and perform the operations described herein.

Although components are described with respect to functional limitations, it should be understood that the particular implementations necessarily include the use of particular computing hardware. It should also be understood that in some embodiments certain of the components described herein include similar or common hardware. For example, in some embodiments two sets of circuitry both leverage use of the same processor(s), memory(ies), circuitry(ies), and/or the like to perform their associated functions such that duplicate hardware is not required for each set of circuitry.

200 140 110 160 200 In various embodiments, such as computing apparatusof a searching optimization system, the plurality of computing devices, and/or the one or more user devicesmay refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablets, phablets, notebooks, laptops, distributed systems, servers, or the like, and/or any combination of devices or entities adapted to perform the functions, operations, and/or processes described herein. Such functions, operations, and/or processes may include, for example, transmitting, receiving, operating on, processing, displaying, storing, determining, creating/generating, monitoring, evaluating, comparing, and/or similar terms used herein. In one embodiment, these functions, operations, and/or processes can be performed on data, content, information, and/or similar terms used herein. In this regard, the apparatusembodies a particular, specially configured computing entity transformed to enable the specific operations described herein and provide the specific advantages associated therewith, as described herein.

202 202 200 200 202 202 Processoror processor circuitrymay be embodied in a number of different ways. In various embodiments, the use of the terms “processor” should be understood to include a single core processor, a multi-core processor, multiple processors internal to the apparatus, and/or one or more remote or “cloud” processor(s) external to the apparatus. In some example embodiments, processormay include one or more processing devices configured to perform independently. Alternatively, or additionally, processormay include one or more processor(s) configured in tandem via a bus to enable independent execution of operations, instructions, pipelining, and/or multithreading.

202 204 202 202 202 202 202 In an example embodiment, the processormay be configured to execute instructions stored in the memoryor otherwise accessible to the processor. Alternatively, or additionally, the processormay be configured to execute hard-coded functionality. As such, whether configured by hardware or software methods, or by a combination thereof, processormay represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to embodiments of the present disclosure while configured accordingly. Alternatively, or additionally, processormay be embodied as an executor of software instructions, and the instructions may specifically configure the processorto perform the various algorithms embodied in one or more operations described herein when such instructions are executed. In some embodiments, the processorincludes hardware, software, firmware, and/or a combination thereof that performs one or more operations described herein.

202 204 200 In some embodiments, the processor(and/or co-processor or any other processing circuitry assisting or otherwise associated with the processor) is/are in communication with the memoryvia a bus for passing information among components of the apparatus.

204 204 204 204 200 Memoryor memory circuitrymay be non-transitory and may include, for example, one or more volatile and/or non-volatile memories. In some embodiments, the memoryincludes or embodies an electronic storage device (e.g., a computer readable storage medium). In some embodiments, the memoryis configured to store information, data, content, applications, instructions, or the like, for enabling an apparatusto carry out various operations and/or functions in accordance with example embodiments of the present disclosure.

206 200 206 206 202 206 206 202 206 204 206 Input/output circuitrymay be included in the apparatus. In some embodiments, input/output circuitrymay provide output to the user and/or receive input from a user. The input/output circuitrymay be in communication with the processorto provide such functionality. The input/output circuitrymay comprise one or more user interface(s). In some embodiments, a user interface may include a display that comprises the interface(s) rendered as a web user interface, an application user interface, a user device, a backend system, or the like. In some embodiments, the input/output circuitryalso includes a keyboard, a mouse, a joystick, a touch screen, touch areas, soft keys a microphone, a speaker, or other input/output mechanisms. The processorand/or input/output circuitrycomprising the processor may be configured to control one or more operations and/or functions of one or more user interface elements through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor (e.g., memory, and/or the like). In some embodiments, the input/output circuitryincludes or utilizes a user-facing application to provide input/output functionality to a display associated with a user.

208 200 208 200 208 208 208 208 200 Communications circuitrymay be included in the apparatus. The communications circuitrymay include any means such as a device or circuitry embodied in either hardware or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device, circuitry, or module in communication with the apparatus. In some embodiments the communications circuitryincludes, for example, a network interface for enabling communications with a wired or wireless communications network. Additionally or alternatively, the communications circuitrymay include one or more network interface card(s), antenna(s), bus(es), switch(es), router(s), modem(s), and supporting hardware, firmware, and/or software, or any other device suitable for enabling communications via one or more communications network(s). In some embodiments, the communications circuitrymay include circuitry for interacting with an antenna(s) and/or other hardware or software to cause transmission of signals via the antenna(s) and/or to handle receipt of signals received via the antenna(s). In some embodiments, the communications circuitryenables transmission to and/or receipt of data from a user device and/or other external computing device(s) in communication with the apparatus.

212 200 212 212 110 160 150 212 110 160 150 200 Data intake circuitrymay be included in the apparatus. The data intake circuitrymay include hardware, software, firmware, and/or a combination thereof, designed and/or configured to capture, receive, request, and/or otherwise gather data. In some embodiments, the data intake circuitryincludes hardware, software, firmware, and/or a combination thereof, that communicates with one or more components of the plurality of computing devices, the one or more user devices, and/or the one or more databasesto receive particular data. Additionally or alternatively, in some embodiments, the data intake circuitryincludes hardware, software, firmware, and/or a combination thereof, that retrieves particular data associated with plurality of computing devices, the one or more user devices, and/or the one or more databasesfrom one or more data repository/repositories accessible to the apparatus.

210 200 210 210 210 210 AI and machine learning circuitrymay be included in the apparatus. The AI and machine learning circuitrymay include hardware, software, firmware, and/or a combination thereof designed and/or configured to request, receive, process, generate, and transmit data, datasets, data structures, control signals, and electronic information for training and executing a trained AI and machine learning model configured to facilitating the operations and/or functionalities described herein. For example, in some embodiments the AI and machine learning circuitryincludes hardware, software, firmware, and/or a combination thereof, that identifies training data and/or utilizes such training data for training a particular machine learning model, AI, and/or other model to generate particular output data based at least in part on learnings from the training data. Additionally or alternatively, in some embodiments, the AI and machine learning circuitryincludes hardware, software, firmware, and/or a combination thereof, that embodies or retrieves a trained machine learning model, AI and/or other specially configured model utilized to process inputted data. Additionally or alternatively, in some embodiments, the AI and machine learning circuitryincludes hardware, software, firmware, and/or a combination thereof that processes received data utilizing one or more algorithm(s), function(s), subroutine(s), and/or the like, in one or more pre-processing and/or subsequent operations that need not utilize a machine learning or AI model.

214 200 214 200 214 214 214 214 200 Data output circuitrymay be included in the apparatus. The data output circuitrymay include hardware, software, firmware, and/or a combination thereof, that configures and/or generates an output based at least in part on data processed by the apparatus. In some embodiments, the data output circuitryincludes hardware, software, firmware, and/or a combination thereof, that generates a particular report based at least in part on the processed data, for example where the report is generated based at least in part on a particular reporting protocol. Additionally or alternatively, in some embodiments, the data output circuitryincludes hardware, software, firmware, and/or a combination thereof, that configures a particular output data object, output data file, and/or user interface for storing, transmitting, and/or displaying. For example, in some embodiments, the data output circuitrygenerates and/or specially configures a particular data output for transmission to another system sub-system for further processing. Additionally or alternatively, in some embodiments, the data output circuitryincludes hardware, software, firmware, and/or a combination thereof, that causes rendering of a specially configured user interface based at least in part on data received by and/or processing by the apparatus.

202 214 202 214 202 214 210 202 202 210 In some embodiments, two or more of the sets of circuitries-are combinable. Alternatively, or additionally, one or more of the sets of circuitry-perform some or all of the operations and/or functionality described herein as being associated with another circuitry. In some embodiments, two or more of the sets of circuitry-are combined into a single module embodied in hardware, software, firmware, and/or a combination thereof. For example, in some embodiments, one or more of the sets of circuitry, for example the AI and machine learning circuitry, may be combined with the processor, such that the processorperforms one or more of the operations described herein with respect the AI and machine learning circuitry.

1 4 FIGS.- 140 110 110 140 110 With reference to, in some embodiments, the searching optimization systemmay be configured to receive one or more datasets from a user. In some embodiments, the user may be associated with at least one of the plurality of computing devices. In this regard, for example, the plurality of computing devicesmay be configured to transmit the one or more datasets to the searching optimization system. In some embodiments, the one or more datasets may comprise one or more documents. For example, if the computing device of the plurality of computing devicesassociated with the user comprises a document management system, the one or more datasets may include documents associated with the document management system, such as design control documents, complaint forms, and/or the like.

140 302 302 150 150 302 302 150 150 302 150 302 302 140 3 FIG. 3 FIG. In some embodiments, the searching optimization systemmay be configured to store at least a portion of the one or more datasets in a first shard of a plurality of shards. In this regard, for example, the first shard of the plurality of shardsmay be a shard associated with the one or more databases(e.g., the one or more databasesare sharded databases). For example, as illustrated in, the first shard of the plurality of shardsmay be one of four shards in the plurality of shardsin the one or more databases. Although depicted inas the one or more databaseshaving four shards in the plurality of shards, it would be understood to one skilled in the field to which this disclosure pertains that the one or more databasesmay have a plurality of shardswith more than four shards and/or less than four shards. For example, the plurality of shardsmay include three shards or five shards. In some embodiments, the searching optimization systemmay be configured to store at least a portion of the one or more datasets in the first shard during a first time period.

302 302 140 302 302 140 In some embodiments, the first shard of the plurality of shardsmay include shard data. In some embodiments, the shard data may include at least a portion of the one or more datasets (e.g., once at least a portion of the one or more datasets has been stored in the first shard). For example, the shard data may include all of the one or more datasets (e.g., the entirety of the one or more datasets are stored in the first shard of the plurality of shards). As another example, the shard data may only include a portion of the one or more datasets. In this regard, for example, the searching optimization systemmay be configured to store a first portion of the one or more datasets in a first shard of the plurality of shardsand a second portion of the one or more datasets in a second shard of the plurality of shards(e.g., the searching optimization systemis configured to shard the one or more datasets). In some embodiments, the shard data may include at least a portion of the one or more datasets and other datasets. For example, the shard data may include at least a portion of the one or more datasets received from the user and other datasets received from the user and/or other users. In some embodiments, the shard data may be associated with a data size. In this regard, for example, the data size may indicate the size of the shard data. For example, the data size may be twenty gigabytes.

140 140 140 140 In some embodiments, the searching optimization systemmay be configured to determine that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, the first data size threshold may be set by the searching optimization system. For example, the searching optimization systemmay set the first data size threshold as twenty five gigabytes. In this regard, for example, the searching optimization systemmay determine that the data size of the shard data is greater than the first data size threshold when the data size is greater than twenty five gigabytes.

302 In some embodiments, the first shard of the plurality of shardsmay be associated with a first data storage size. In some embodiments, the first data storage size may indicate a maximum amount of data the first shard may store. In some embodiments, the first data storage size may be greater than the first data size threshold. For example, the first data storage size may be 50 gigabytes and the first data size threshold may be twenty five gigabytes.

140 302 140 140 302 140 140 140 140 In some embodiments, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, the searching optimization systemmay be configured to transfer at least a portion of the shard data in the first shard to a second shard in the plurality of shards. For example, if the first data size threshold is twenty five gigabytes and the searching optimization systemdetermines that the data size of the shard data in the first shard is greater than twenty five gigabytes, the searching optimization systemmay transfer at least a portion of the shard data in the first shard to a second shard in the plurality of shards. In some embodiments, the searching optimization systemmay transfer enough of the shard data in the first shard to the second shard such that the data size of the shard data in the first shard is equal to or less than the first data size threshold. For example, if the first data size threshold is twenty five gigabytes and the data size of the shard data is thirty gigabytes, the searching optimization systemmay transfer at least five gigabytes of the shard data in the first shard to the second shard. Additionally or alternatively, the searching optimization systemmay transfer enough of the shard data in the first shard to the second shard such that the data size of the shard data in the first shard and a data size of shard data in the second shard may be approximately equal after the transfer. For example, if, before the transfer, the data size of the shard data in the first shard is thirty gigabytes and the data size of the shard data in the second shard is ten gigabytes, the searching optimization systemmay transfer ten gigabytes of the shard data in the first shard to the second shard such that after the transfer the data size of the shard data in the first shard is twenty gigabytes and the data size of the shard data in the second shard is twenty gigabytes.

302 In some embodiments, transferring at least a portion of the shard data in the first shard to the second shard of the plurality of shardsmay occur during a third time period. In some embodiments, the third time period may occur after the first time period. That is, transferring at least a portion of the shard data in the first shard to the second shard may occur after at least a portion of the one or more datasets have been stored in the first shard.

140 140 140 140 210 In some embodiments, the third time period is based at least in part on historical activity data associated with the user. In some embodiments, the historical activity data may indicate that a user is inactive during the third time period. Said differently, the searching optimization systemmay be configured to collect activity data indicating time periods when the user is active and inactive and, based on an analysis of historical activity data (e.g., previously collected activity data), the searching optimization systemmay be configured to determine that the user will be inactive during the third time period. In this regard, the searching optimization systemmay be configured to transfer at least a portion of the shard data in the first shard to the second shard without causing any interruption to the user (e.g., the user is not impacted by the transferring). In some embodiments, the searching optimization systemmay be configured to analyze the historical activity data to determine that a user is inactive during the third time period using one or more of a statistical model, an algorithmic model, and/or a machine learning model (e.g., using AI and machine learning circuitry).

140 140 In some embodiments, the third time period may be based at least in part on user specified activity data. In some embodiments, the user specified activity data may indicate a time period (e.g., the third time period) in which the user has indicated that the searching optimization systemshould transfer at least a portion of the shard data in the first shard to the second shard. In this regard, for example, the user specified activity data indicates a time period in which the user has indicated that the user will be inactive. For example, if the user specified activity data indicates that the third time period should be on a Saturday from 1 am to 2 am, the searching optimization systemwill transfer at least a portion of the shard data in the first shard to the second shard on a Saturday from 1 am to 2 am.

110 140 140 140 In some embodiments, the third time period may be based at least in part on user location data. In some embodiments, the user location data may indicate a location associated with the user (e.g., a location associated with a computing device of the plurality of computing devices). In this regard, for example, the searching optimization systemmay use the user location data to determine a time period (e.g., the third time period) when a user is inactive (e.g., the user location data is used to determine when it is nighttime and/or the weekend at a user's location). In this regard, for example, the searching optimization systemmay be configured to transfer at least a portion of the shard data in the first shard to the second shard when the user location data indicates that a user is inactive. For example, if user location data associated with a user indicates that a user is located in the Eastern Time Zone, the searching optimization systemmay be configured to transfer at least a portion of the shard data in the first shard to the second shard when it is nighttime and/or when it is the weekend in the Eastern Time Zone.

302 150 302 150 302 150 302 150 150 150 In some embodiments, the first shard of the plurality of shardsmay be associated with a first database of the one or more databasesand the second shard of the plurality of shardsmay be associated with the first database of the one or more databases. In some embodiments, the first shard of the plurality of shardsmay be associated with the first database of the one or more databasesand the second shard of the plurality of shardsmay be associated with a second database of the one or more databases. That is, in some embodiments, the first shard and the second shard may each be associated with the same database in the one or more databaseswhile, in other embodiments, the first shard and the second shard may each be associated with a different database in the one or more databases.

140 140 140 302 In some embodiments, the searching optimization systemmay be configured to receive a first query from the user to access at least a portion of the shard data. For example, the first query may be a search by the user of the shard data (e.g., a text search). In some embodiments, the searching optimization systemmay be configured to receive the first query during a second time period. In some embodiments, the second time period may occur after the first time period (e.g., after the one or more datasets have been stored in the first shard). In some embodiments, the second time period may occur before the third time period (e.g., before the searching optimization systemhas transferred at least a portion of the shard data in the first shard to the second shard of the plurality of shards).

140 140 140 302 In some embodiments, the searching optimization systemmay be configured to receive a second query from the user to access at least a portion of the shard data. For example, the second query may be a search by the user of the shard data (e.g., a text search). In some embodiments, the searching optimization systemmay be configured to receive the second query during a fourth time period. In some embodiments, the fourth time period may occur after the third time period (e.g., after the searching optimization systemhas transferred at least a portion of the shard data in the first shard to the second shard of the plurality of shards).

140 140 140 In some embodiments, the searching optimization systemmay be configured to return search results in response to the first query. For example, the search results may be a portion of the shard data (e.g., a portion of the shard data that is relevant to the first query). In some embodiments, the first query may be associated with a first latency. In this regard, the first latency may be an amount of time that it takes for the searching optimization systemto return the search results in response to the first query. For example, it may take the searching optimization systemten milliseconds to return the search results in response to the first query.

140 140 140 In some embodiments, the searching optimization systemmay be configured to return search results in response to the second query. For example, the search results may be a portion of the shard data (e.g., a portion of the shard data that is relevant to the second query). In some embodiments, the second query may be associated with a second latency. In this regard, the second latency may be an amount of time that it takes for the searching optimization systemto return the search results in response to the second query. For example, it may take the searching optimization systemfour milliseconds to return the search results in response to the second query.

140 In some embodiments, the first latency may be greater than the second latency. That is, it may take the searching optimization systemlonger to return the search results in response to the first query than it does to return the search results in response to the second query. For example, the first latency may be ten milliseconds and the second latency may be four milliseconds. In this regard, for example, the first latency may be greater than the second latency because the greater the data size of shard data in the first shard, the greater amount of time it may take to return the search results in response to the first query. Additionally or alternatively, the first latency may be greater than the second latency because the closer the data size of shard data in the first shard is to the first data storage size, the greater amount of time it may take to return the search results in response to the first query.

140 400 400 400 400 400 400 400 140 140 302 In some embodiments, the searching optimization systemmay be configured to cause a user interfaceto be displayed. In some embodiments, the user interfacemay be configured to display a plurality of latencies associated with a plurality of queries during the second and fourth time period. In this regard, the user interfacemay be configured to display a plurality of latencies associated with a plurality of queries during the second time period. For example, the user interfacemay be configured to display the first latency (e.g., ten milliseconds). Additionally or alternatively, the user interfacemay be configured to display a plurality of latencies associated with a plurality of queries during the fourth time period. For example, the user interfacemay be configured to display the second latency (e.g., four milliseconds). In this regard, the user interfacemay enable a user of the searching optimization systemto monitor latencies associated with queries before and after the searching optimization systemhas transferred at least a portion of the shard data in the first shard to the second shard in the plurality of shards.

400 400 400 140 140 140 In some embodiments, the user interfacemay be configured to display the first time period and the third time period. In some embodiments, the first time period and/or the third time period displayed on the user interfacemay not include any latencies associated with queries (e.g., because the searching optimization does not receive and/or process any queries when storing the one or more datasets in the first shard and/or when transferring at least a portion of the shard data from the first shard to the second shard). In this regard, the user interfacemay enable a user of the searching optimization systemto monitor the amount of time it takes for the searching optimization systemto store at least a portion of the one or more datasets in the first shard and/or the amount of time it takes for the searching optimization systemto transfer at least a portion of the shard data in the first shard to the second shard.

140 302 140 140 140 In some embodiments, the searching optimization systemmay be configured to determine that a second data size of second shard data in a third shard of the plurality of shardsis less than a second data size threshold. In some embodiments, the second data size threshold may be set by the searching optimization system. For example, the searching optimization systemmay set the second data size threshold as five gigabytes. In this regard, for example, the searching optimization systemmay be configured to determine that the second data size of the second shard data is less than the second data size threshold when the second data size is less than five gigabytes.

140 302 140 140 302 140 140 140 302 140 302 140 302 140 140 In some embodiments, in response to the determination that the second data size of the second shard data in the third shard is less than the second data size threshold, the searching optimization systemmay be configured to transfer at least a portion of the second shard data in the third shard to a fourth shard in the plurality of shards. For example, if the second data size threshold is five gigabytes and the searching optimization systemdetermines that the second data size the second shard data in the third shard is less than five gigabytes, the searching optimization systemmay transfer at least a portion of the second shard data in the third shard to a fourth shard in the plurality of shards. In some embodiments, the searching optimization systemmay transfer all of the second shard data in the third shard to the fourth shard. For example, if the second data size threshold is five gigabytes and the second data size of the second shard data is four gigabytes, the searching optimization systemmay transfer all four gigabytes of the second shard data in the third shard to the fourth shard. In this regard, for example, the searching optimization systemmay be configured to efficiently utilize the plurality of shardsin a cost effective manner. Said differently, if the searching optimization systemdetermines that a shard in the plurality of shards(e.g., the third shard) is only using a portion of the storage size of a shard (e.g., the second data size of the second shard data in the third shard is less than the second data size threshold), the searching optimization systemmay transfer shard data to another shard in the plurality of shardsthat has free storage space. In this way, the searching optimization systemmay be configured to consolidate shard data from shards that are each only storing a minimal amount of shard data (e.g., shards having shard data with a data size less than the second data size threshold) into a fewer number of shards, enabling the searching optimization systemto reduce costs (e.g., less shards have to be maintained).

5 FIG. 5 FIG. 500 140 140 Referring now to, a flowchart providing an example computer-implemented methodis illustrated. In this regard,illustrates operations that may be performed by the searching optimization systemand/or components of the searching optimization system.

502 500 As shown in block, the computer-implemented methodmay include receiving one or more datasets from a user. As described above, in some embodiments, the user may be associated with at least one of the plurality of computing devices. In this regard, for example, the plurality of computing devices may be configured to transmit the one or more datasets. In some embodiments, the one or more datasets may comprise one or more documents. For example, if the computing device of the plurality of computing devices associated with the user comprises a document management system, the one or more datasets may include documents associated with the document management system, such as design control documents, complaint forms, and/or the like.

504 500 As shown in block, the computer-implemented methodmay include storing at least a portion of the one or more datasets in a first shard during a first time period. As described above, in some embodiments, the first shard may be one of a plurality of shards. In this regard, for example, the first shard of the plurality of shards may be a shard associated with the one or more databases (e.g., the one or more databases are sharded databases). For example, the first shard of the plurality of shards may be one of four shards in the plurality of shards in the one or more databases.

In some embodiments, the first shard of the plurality of shards may include shard data. In some embodiments, the shard data may include at least a portion of the one or more datasets (e.g., once at least a portion of the one or more datasets has been stored in the first shard). For example, the shard data may include all of the one or more datasets (e.g., the entirety of the one or more datasets are stored in the first shard of the plurality of shards). As another example, the shard data may only include a portion of the one or more datasets. In this regard, a first portion of the one or more datasets may be stored in a first shard of the plurality of shards and a second portion of the one or more datasets may be stored in a second shard of the plurality of shards (e.g., the one or more datasets may be sharded). In some embodiments, the shard data may include at least a portion of the one or more datasets and other datasets. For example, the shard data may include at least a portion of the one or more datasets received from the user and other datasets received from the user and/or other users. In some embodiments, the shard data may be associated with a data size. In this regard, for example, the data size may indicate the size of the shard data. For example, the data size may be twenty gigabytes.

506 500 As shown in block, the computer-implemented methodmay include determining that the data size of the shard data in the first shard is greater than a first data size threshold. As described above, in some embodiments, it may be determined that the data size of the shard data in the first shard is greater than a first data size threshold. In some embodiments, the first data size threshold may be set. For example, the first data size threshold may be set as twenty five gigabytes. In this regard, for example, it may be determined that the data size of the shard data is greater than the first data size threshold when the data size is greater than twenty five gigabytes.

In some embodiments, the first shard of the plurality of shards may be associated with a first data storage size. In some embodiments, the first data storage size may indicate a maximum amount of data the first shard may store. In some embodiments, the first data storage size may be greater than the first data size threshold. For example, the first data storage size may be 50 gigabytes and the first data size threshold may be twenty five gigabytes.

508 500 As shown in block, the computer-implemented methodmay include, in response to the determination that the data size of the shard data in the first shard is greater than the first data size threshold, transferring at least a portion of the shard data in the first shard to a second shard. As described above, for example, if the first data size threshold is twenty five gigabytes and it is determined that the data size of the shard data in the first shard is greater than twenty five gigabytes, at least a portion of the shard data in the first shard may be transferred to a second shard in the plurality of shards. In some embodiments, enough of the shard data in the first shard may be transferred to the second shard such that the data size of the shard data in the first shard is equal to or less than the first data size threshold. For example, if the first data size threshold is twenty five gigabytes and the data size of the shard data is thirty gigabytes, at least five gigabytes of the shard data in the first shard may be transferred to the second shard. Additionally or alternatively, enough of the shard data in the first shard may be transferred to the second shard such that the data size of the shard data in the first shard and a data size of shard data in the second shard may be approximately equal after the transfer. For example, if, before the transfer, the data size of the shard data in the first shard is thirty gigabytes and the data size of the shard data in the second shard is ten gigabytes, ten gigabytes of the shard data in the first shard may be transferred to the second shard such that after the transfer the data size of the shard data in the first shard is twenty gigabytes and the data size of the shard data in the second shard is twenty gigabytes.

In some embodiments, transferring at least a portion of the shard data in the first shard to the second shard of the plurality of shards may occur during a third time period. In some embodiments, the third time period may occur after the first time period. That is, transferring at least a portion of the shard data in the first shard to the second shard may occur after at least a portion of the one or more datasets have been stored in the first shard.

In some embodiments, the third time period is based at least in part on historical activity data associated with the user. In some embodiments, the historical activity data may indicate that a user is inactive during the third time period. Said differently, activity data indicating time periods when the user is active and inactive may be collected and, based on an analysis of historical activity data (e.g., previously collected activity data), it may be determined that the user will be inactive during the third time period. In this regard, at least a portion of the shard data in the first shard may be transferred to the second shard without causing any interruption to the user (e.g., the user is not impacted by the transferring). In some embodiments, the historical activity data may be analyzed to determine that a user is inactive during the third time period using one or more of a statistical model, an algorithmic model, and/or a machine learning model (e.g., using AI and machine learning circuitry).

In some embodiments, the third time period may be based at least in part on user specified activity data. In some embodiments, the user specified activity data may indicate a time period (e.g., the third time period) in which the user has indicated that at least a portion of the shard data in the first shard should be transferred to the second shard. In this regard, for example, the user specified activity data indicates a time period in which the user has indicated that the user will be inactive. For example, if the user specified activity data indicates that the third time period should be on a Saturday from 1 am to 2 am, at least a portion of the shard data in the first shard to the second shard may be transferred on a Saturday from 1 am to 2 am.

In some embodiments, the third time period may be based at least in part on user location data. In some embodiments, the user location data may indicate a location associated with the user (e.g., a location associated with a computing device of the plurality of computing devices). In this regard, for example, the user location data may be used to determine a time period (e.g., the third time period) when a user is inactive (e.g., the user location data is used to determine when it is nighttime and/or the weekend at a user's location). In this regard, for example, at least a portion of the shard data in the first shard may be transferred to the second shard when the user location data indicates that a user is inactive. For example, if a user's user location data indicates that a user is located in the Eastern Time Zone, at least a portion of the shard data in the first shard may be transferred to the second shard when it is nighttime and/or when it is the weekend in the Eastern Time Zone.

In some embodiments, the first shard of the plurality of shards may be associated with a first database of the one or more databases and the second shard of the plurality of shards may be associated with the first database of the one or more databases. In some embodiments, the first shard of the plurality of shards may be associated with the first database of the one or more databases and the second shard of the plurality of shards may be associated with a second database of the one or more databases. That is, in some embodiments, the first shard and the second shard may each be associated with the same database in the one or more databases while, in other embodiments, the first shard and the second shard may each be associated with a different database in the one or more databases.

510 500 As shown in block, the computer-implemented methodmay optionally include receiving a first query from the user to access at least a portion of the shard data during a second time period. As described above, for example, the first query may be a search by the user of the shard data (e.g., a text search). In some embodiments, the first query may be received during a second time period. In some embodiments, the second time period may occur after the first time period (e.g., after the one or more datasets have been stored in the first shard). In some embodiments, the second time period may occur before the third time period.

512 500 As shown in block, the computer-implemented methodmay optionally include receiving a second query from the user to access at least a portion of the shard data during a fourth time period. As described above, for example, the second query may be a search by the user of the shard data (e.g., a text search). In some embodiments, the second query may be received during a fourth time period. In some embodiments, the fourth time period may occur after the third time period.

In some embodiments, search results may be returned in response to the first query. For example, the search results may be a portion of the shard data (e.g., a portion of the shard data that is relevant to the first query). In some embodiments, the first query may be associated with a first latency. In this regard, the first latency may be an amount of time that it takes for the search results to be returned in response to the first query. For example, it may take ten milliseconds to return the search results in response to the first query.

In some embodiments, search results may be returned in response to the second query. For example, the search results may be a portion of the shard data (e.g., a portion of the shard data that is relevant to the second query). In some embodiments, the second query may be associated with a second latency. In this regard, the second latency may be an amount of time that it takes for the search results to be returned in response to the second query. For example, it may take four milliseconds to return the search results in response to the second query.

In some embodiments, the first latency may be greater than the second latency. That is, it may take longer to return the search results in response to the first query than it does to return the search results in response to the second query. For example, the first latency may be ten milliseconds and the second latency may be four milliseconds. In this regard, for example, the first latency may be greater than the second latency because the greater the data size of shard data in the first shard, the greater amount of time it may take to return the search results in response to the first query. Additionally or alternatively, the first latency may be greater than the second latency because the closer the data size of shard data in the first shard is to the first data storage size, the greater amount of time it may take to return the search results in response to the first query.

In some embodiments, a user interface may be displayed. In some embodiments, the user interface may be configured to display a plurality of latencies associated with a plurality of queries during the second and fourth time period. In this regard, the user interface may be configured to display a plurality of latencies associated with a plurality of queries during the second time period. For example, the user interface may be configured to display the first latency (e.g., ten milliseconds). Additionally or alternatively, the user interface may be configured to display a plurality of latencies associated with a plurality of queries during the fourth time period. For example, the user interface may be configured to display the second latency (e.g., four milliseconds). In this regard, the user interface may enable a user to monitor latencies associated with queries before and after at least a portion of the shard data in the first shard has been transferred to the second shard in the plurality of shards.

514 500 As shown in block, the computer-implemented methodmay optionally include determining that a second data size of second shard data in a third shard is less than a second data size threshold. As described above, in some embodiments, the second data size threshold may be set. For example, the second data size threshold may be set as five gigabytes. In this regard, for example, it may be determined that the second data size of the second shard data is less than the second data size threshold when the second data size is less than five gigabytes.

516 500 As shown in block, the computer-implemented methodmay optionally include, in response to the determination that the second data size of the second shard data in the third shard is less than the second data size threshold, transferring at least a portion of the second shard data in the third shard to a fourth shard. As described above, for example, if the second data size threshold is five gigabytes it is determined that the second data size the second shard data in the third shard is less than five gigabytes, at least a portion of the second shard data in the third shard may be transferred to a fourth shard in the plurality of shards. In some embodiments, all of the second shard data in the third shard may be transferred to the fourth shard. For example, if the second data size threshold is five gigabytes and the second data size of the second shard data is four gigabytes, all four gigabytes of the second shard data in the third shard may be transferred to the fourth shard. In this regard, for example, the plurality of shards may be efficiently utilized in a cost effective manner. Said differently, if it is determined that a shard in the plurality of shards (e.g., the third shard) is only using a portion of the storage size of a shard (e.g., the second data size of the second shard data in the third shard is less than the second data size threshold), shard data may be transferred to another shard in the plurality of shards that has free storage space. In this way, shard data may be consolidated from shards that are each only storing a minimal amount of shard data (e.g., shards having shard data with a data size less than the second data size threshold) into a fewer number of shards, enabling cost reduction (e.g., less shards have to be maintained).

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of teachings presented in the foregoing descriptions and the associated drawings. Although the figures only show certain components of the apparatus and systems described herein, it is understood that various other components may be used in conjunction with the system. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, the steps in the method described above may not necessarily occur in the order depicted in the accompanying diagrams, and in some cases one or more of the steps depicted may occur substantially simultaneously, or additional steps may be involved. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

While various embodiments in accordance with the principles disclosed herein have been shown and described above, modifications thereof may be made by one skilled in the art without departing from the spirit and the teachings of the disclosure. The embodiments described herein are representative only and are not intended to be limiting. Many variations, combinations, and modifications are possible and are within the scope of the disclosure. Alternative embodiments that result from combining, integrating, and/or omitting features of the embodiment(s) are also within the scope of the disclosure. Accordingly, the scope of protection is not limited by the description set out above.

Additionally, the section headings used herein are provided for consistency with the suggestions under 37 C.F.R. 1.77 or to otherwise provide organizational cues. These headings shall not limit or characterize the invention(s) set out in any claims that may issue from this disclosure.

Use of broader terms such as “comprises,” “includes,” and “having” should be understood to provide support for narrower terms such as “consisting of,” “consisting essentially of,” and “comprised substantially of” Use of the terms “optionally,” “may,” “might,” “possibly,” and the like with respect to any element of an embodiment means that the element is not required, or alternatively, the element is required, both alternatives being within the scope of the embodiment(s). Also, references to examples are merely provided for illustrative purposes, and are not intended to be exclusive.