System, Method, and Apparatus for AI-Powered Insight Extraction from Video Content

The present application claims priority under 35 U.S.C. § 119 (e) to U.S. Provisional Patent Application No. 63/638,656, filed Apr. 25, 2024, which application is incorporated herein by reference in its entirety.

When dealing with large volumes of data, companies often millions of data artifacts scattered across various platforms, both current and legacy. As a result, these companies, including tech companies, typically have problems finding accurate, up-to-date, data, even as it relates to their own products.

Accordingly, there remains a need in the art for systems and methods for data ingest, analysis, and organization. The present invention meets this need.

In one aspect, a system for providing a database engine includes a data ingestion layer; a data processing and enrichment layer; and a knowledge serving layer; wherein the system is configured to provide an automated source of truth. In some embodiments, the system is configured to ingest and filter data from one or more data sources and automatically validate, catalogue, index, tag, assign a confidence score along with document links, or a combination thereof. In some embodiments, the data includes unstructured data.

In some embodiments, the data ingestion layer includes at least one connector coupling the system to the one or more data sources and a data repository.

In some embodiments, the data processing and enrichment layer comprises at least one of a de-duplication engine; a conflict resolution module; a trustworthiness scoring module; and a tagging and indexing service.

In some embodiments, the de-duplication engine comprises a machine-learning algorithm. In some embodiments, the machine-learning algorithm is configured to de-duplicate the data at a factoid level. In some embodiments, the de-duplication engine is configured to combine the machine-learning algorithm with at least one other de-duplication approach.

In some embodiments, the conflict resolution module is configured to assess conflicting information and determine a most likely truth. In some embodiments, the conflict resolution module comprises a machine learning algorithm. In some embodiments, the machine learning algorithm is trained to determine the most likely truth through at least one of analyzing source credibility, analyzing source recency, and corroborating across multiple sources. In some embodiments, the trustworthiness scoring module is configured to assign a confidence score to the most likely truth. In some embodiments, the confidence score id based upon at least one of source reliability, data age, level of agreement across sources, and/or the outcome of the conflict resolution process.

In some embodiments, the tagging and indexing service is configured to automatically extract metadata from the data. In some embodiments, the tagging and indexing service extracts the metadata using natural language processing.

In some embodiments, the knowledge serving layer comprises a search application programming interface (API), the API configured to provide an interface for querying indexed data from the data processing and enrichment layer. In some embodiments, the search API is configured to provide search results with at least one of an identified “truth,” a confidence score, and a link to original source artifacts in the data. In some embodiments, the knowledge serving layer further comprises a training interface configured to enable review of low-confidence data.

The instant invention is most clearly understood with reference to the following definitions.

As used herein, the singular form “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.

Unless specifically stated or obvious from context, as used herein, the term “about” is understood as within a range of normal tolerance in the art, for example within 2 standard deviations of the mean. “About” can be understood as within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the stated value. Unless otherwise clear from context, all numerical values provided herein are modified by the term about.

As used in the specification and claims, the terms “comprises,” “comprising,” “containing,” “having,” and the like can have the meaning ascribed to them in U.S. patent law and can mean “includes,” “including,” and the like.

Unless specifically stated or obvious from context, the term “or,” as used herein, is understood to be inclusive.

The terms “proximal” and “distal” can refer to the position of a portion of a device relative to the remainder of the device or the opposing end as it appears in the drawing. The proximal end can be used to refer to the end manipulated by the user. The distal end can be used to refer to the end of the device that is inserted and advanced and is furthest away from the user. As will be appreciated by those skilled in the art, the use of proximal and distal could change in another context, e.g., the anatomical context in which proximal and distal use the patient as reference, or where the entry point is distal from the user.

The terms “data,” “content,” “digital content,” “digital content object,” “signal,” “information,” and similar terms may be used interchangeably to refer to data capable of being transmitted, received, and/or stored in accordance with embodiments of the present disclosure. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the present disclosure. Further, where a computing device is described herein to receive data from another computing device, it will be appreciated that the data may be received directly from another computing device or may be received indirectly via one or more intermediary computing devices, such as, for example, one or more servers, relays, routers, network access points, base stations, hosts, and/or the like, sometimes referred to herein as a “network.” Similarly, where a computing device is described herein to send data to another computing device, it will be appreciated that the data may be transmitted directly to another computing device or may be transmitted indirectly via one or more intermediary computing devices, such as, for example, one or more servers, relays, routers, network access points, base stations, hosts, and/or the like.

Ranges provided herein are understood to be shorthand for all of the values within the range. For example, a range of 1 to 50 is understood to include any number, combination of numbers, or sub-range from the group consisting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, or 50 (as well as fractions thereof unless the context clearly dictates otherwise).

Provided herein are systems configured to enable automated data engines. In some embodiments, the system is configured to provide an automated source of truth. For example, in some embodiments, the system includes a data ingestion layer, a data processing and enrichment layer, and a knowledge serving layer. In some such embodiments, the system ingests and filters unstructured and/or structured data from one or more data sources, and automatically validates, catalogues, indexes, tags, and/or assigns a confidence score along with document links.

Unstructured data sources include, but are not limited to, data storage elements (e.g., cloud storage services, such as Google Drive), email servers, chat programs/platforms, collaborative editing services (e.g., Confluence, other wikis), or any other suitable unstructured data source. Structured data sources include, but are not limited to, any repository that organizes data in a clear, predefined format. For example, in some embodiments, the structured data source includes a structured query language (SQL) database.

The data ingestion layer includes any suitable structure for gathering and/or storing data. In some embodiments, for example, the data ingestion layer includes one or more connectors and a data repository. The connectors can be pre-built (e.g., Google Cloud Integration Connectors) and/or custom (e.g., Google Cloud Functions, Cloud Run) connectors to any suitable data source(s) (e.g., structured or unstructured). In some embodiments, the connectors couple the system to one or more data sources using application programming interfaces (APIs) or other suitable integration methods. The data repository includes any suitable repository for storing raw, unstructured data in its native format, such as, but not limited to, a cloud storage service (e.g., Google Cloud Storage).

The data processing and enrichment layer includes any suitable structure for analyzing disparate data sources, correlating related data sources, deduplicating redundant data elements, deconflicting conflicted data elements, and/or storing processed data elements along with a confidence score of the elements validity. In some embodiments, the data processing and enrichment layer includes a de-duplication engine configured to identify and merge de-duplication elements. For example, in some embodiments, the de-duplication engine utilizes a machine-learning model or other artificial intelligence to identify and merge the de-duplication elements. Additionally or alternatively, in some embodiments, the de-duplication engine combines the machine-learning model or other artificial intelligence with traditional methods (e.g., hashing, semantic similarity analysis using embeddings). The machine-learning model can be pre-trained or custom-trained. The custom-trained models can be trained using any suitable training method, such as, but not limited to, natural language API, text similarity, AutoML, any other suitable training method, and/or combinations thereof.

In contrast to existing methods, where de-duplication is applied to data blocks, data files, and the like, the de-duplication engine described herein is configured to de-duplicate at a factoid level (i.e., the de-duplication elements are factoids). In some such embodiments, the individual factoids are not in identical form, but instead refer to the same fact/element. As an example, when one factoid includes “Lincoln was elected in 1860” and another includes “the president who won the election of 1860 was Abraham Lincoln,” the de-duplication engine recognizes a duplicate factoid and stores a single data element (e.g., “Lincoln elected 1860”). In some embodiments, the single data element is stored with indexing hash for later recall.

In some embodiments, the data processing and enrichment layer also includes a conflict resolution module configured to assess conflicting information and determine the most likely “truth.” For example, in some embodiments, the conflict resolution model includes AI-powered techniques for assessing conflicting information and determining the most likely truth. Suitable AI-powered techniques include, but are not limited to, natural language understanding to analyze content, machine learning models trained on historical conflict resolution data, and/or any other suitable AI-powered technique. In some embodiments, the A I-powered technique for determining the most likely truth includes analyzing source credibility, recency, and corroboration across multiple sources.

In some embodiments, the data processing and enrichment layer further includes a trustworthiness scoring module. Following deduplication and/or conflict resolution, the trustworthiness scoring module assigns a confidence score to the “truth” data. The confidence score can be based upon any suitable factors, such as, but not limited to, source reliability, data age, level of agreement across sources, and/or the outcome of the conflict resolution process. In some embodiments, the confidence score is determined using statistical methods and/or machine learning models.

Additionally or alternatively, in some embodiments, the data processing and enrichment layer includes a tagging and indexing service. In some embodiments, the tagging and indexing service includes natural language processing (NLP) configured to automatically extract relevant information from the data. The NLP includes any suitable NLP for extracting the relevant information. As will be appreciated by those skilled in the art, the relevant information depends upon the data and the particular tagging/indexing desired. For example, in some embodiments, the relevant information includes keywords, entities, and/of topics. Following extraction, the relevant information, or metadata, along with the full text, is indexed for rapid retrieval. The metadata can be indexed in any suitable manner, such as, but not limited to, using a search engine (e.g., Google Cloud Search or Elasticsearch). The metadata, and configurations, can be stored in any suitable database or storage element, such as, but not limited to, Google Cloud Firestore (NoSQL) or Google Cloud SQL (PostgreSQL or MySQL) for storing metadata about ingested data, confidence scores, tags, and platform configurations.

The knowledge serving layer includes a search API configured to provide an interface for querying the indexed data from the data processing and enrichment layer. The interface can be configured for any suitable user and/or other system to query the indexed data. In some embodiments, search results are provided with the identified “truth,” its confidence score, and links back to the original source artifacts in the data lake/repository. Additionally or alternatively, in some embodiments, the knowledge serving layer includes a training interface configured to enable review (e.g., by a subject matter expert) of low-confidence “truth” data. In some embodiments, a threshold for the low-confidence “truth” data is user controlled and can be set to a desired confidence level (e.g., if a user sets the threshold for low-confidence at 70%, the system flags anything with a confidence score below 70%). The threshold for the low-confidence “truth” data includes any desired threshold appropriate for the type of data. In some embodiments, the low-confidence data is flagged for review by an appropriate subject matter expert and/or moved into a review pool. In some embodiments, the review includes providing feedback, correcting inaccuracies, and/or labeling data to further train the deduplication and/or conflict resolution models. The training interface can be provided in any suitable platform, such as, but not limited to, a web-based interfaces.

In some embodiments, the system includes one or more additional modules based upon a desired end-use. For example, in some embodiments, the system includes an AI troubleshooting assistant. In such embodiments, the AI troubleshooting assistant utilizes the source of truth architecture to provide guided self-service troubleshooting with advanced correlation capability. In another example, the system include a bespoke adoption guide generator configured to save customers hours of config guide time by providing step-by-step deployment guides tailored to their unique implementation and integrations. In a further example, the system includes a deal validator. In such embodiments, the deal validator is configured to ensure that a Bill of Materials (BOM) isn't missing anything required for success. Where multiple components, or SK Us can fulfill the same role, recommend the highest margin/rebate/etc. option.

As a result of the modular architecture, the system described herein can be readily adapted to any suitable application, including any application where automated data ingestion, validation, and indexing can be employed. Such applications include, but are not limited to, sales, search services, or any other suitable application. For example, in some embodiments, the system described herein is configured to ingest and filter unstructured and structured data from customer relationship databases (CRM s), product documentation, sales documentation, sales operations, etc. Additionally or alternatively, in some embodiments, the unstructured data includes video files. This data is automatically validated, catalogued, indexed, tagged, and assigned a confidence score along with source document links. The result is an automated Source of Truth (SoT) for product, sales, and technical information key to sales cycle success.

The system described herein can be implemented on any suitable computer system, architecture, or program. Therefore, although various embodiments are discussed below, as will be understood by those skilled in the art, the disclosure is not so limited and includes any other suitable computing platform capable of implementing the system described herein.

Embodiments of the present disclosure may be implemented in various ways, including as computer program products that comprise articles of manufacture. Such computer program products may include one or more software components including, for example, software objects, methods, data structures, or the like. A software component may be coded in any of a variety of programming languages. An illustrative programming language may be a lower-level programming language such as an assembly language associated with a particular hardware framework and/or operating system platform. A software component comprising assembly language instructions may require conversion into executable machine code by an assembler prior to execution by the hardware framework and/or platform. Another example programming language may be a higher-level programming language that may be portable across multiple frameworks. A software component comprising higher-level programming language instructions may require conversion to an intermediate representation by an interpreter or a compiler prior to execution.

Other examples of programming languages include, but are not limited to, a macro language, a shell or command language, a job control language, a script language, a database query, or search language, and/or a report writing language. In one or more example embodiments, a software component comprising instructions in one of the foregoing examples of programming languages may be executed directly by an operating system or other software component without having to be first transformed into another form. A software component may be stored as a file or other data storage construct. Software components of a similar type or functionally related may be stored together such as in a particular directory, folder, or library. Software components may be static (e.g., pre-established or fixed) or dynamic (e.g., created or modified at the time of execution).

A computer program product may include non-transitory computer-readable storage medium storing applications, programs, program modules, scripts, source code, program code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like (also referred to herein as executable instructions, instructions for execution, computer program products, program code, and/or similar terms used herein interchangeably). Such non-transitory computer-readable storage media include all computer-readable media (including volatile and non-volatile media).

In one embodiment, a non-volatile computer-readable storage medium may include a floppy disk, flexible disk, hard disk, solid-state storage (SSS) (e.g., a solid-state drive (SSD)), solid state card (SSC), solid state module (SSM), enterprise flash drive, magnetic tape, or any other non-transitory magnetic medium, and/or the like. A non-volatile computer-readable storage medium may also include a punch card, paper tape, optical mark sheet (or any other physical medium with patterns of holes or other optically recognizable indicia), compact disc read only memory (CD-ROM), compact disc-rewritable (CD-RW), digital versatile disc (DVD), Blu-ray disc (BD), any other non-transitory optical medium, and/or the like. Such a non-volatile computer-readable storage medium may also include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory (e.g., Serial, NAND, NOR, and/or the like), multimedia memory cards (MMC), secure digital (SD) memory cards, SmartMedia cards, CompactFlash (CF) cards, Memory Sticks, and/or the like. Further, a non-volatile computer-readable storage medium may also include conductive-bridging random access memory (CBRAM), phase-change random access memory (PRAM), ferroelectric random-access memory (FeRAM), non-volatile random-access memory (NVRAM), magnetoresistive random-access memory (MRAM), resistive random-access memory (RRAM), Silicon-Oxide-Nitride-Oxide-Silicon memory (SONOS), floating junction gate random access memory (FJGRAM), Millipede memory, racetrack memory, and/or the like.

In one embodiment, a volatile computer-readable storage medium may include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), fast page mode dynamic random access memory (FPM DRAM), extended data-out dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), double data rate type two synchronous dynamic random access memory (DDR2SDRAM), double data rate type three synchronous dynamic random access memory (DDR3SDRAM), Rambus dynamic random access memory (RDRAM), Twin Transistor RAM (TTRAM), Thyristor RAM (T-RAM), Zero-capacitor (Z-RAM), Rambus in-line memory module (RIMM), dual in-line memory module (DIMM), single in-line memory module (SIMM), video random access memory (VRAM), cache memory (including various levels), flash memory, register memory, and/or the like. It will be appreciated that where embodiments are described to use a computer-readable storage medium, other types of computer-readable storage media may be substituted for or used in addition to the computer-readable storage media described above.

As should be appreciated, various embodiments of the present disclosure may also be implemented as methods, apparatuses, systems, computing devices, computing entities, and/or the like. As such, embodiments of the present disclosure may take the form of an apparatus, system, computing device, computing entity, and/or the like executing instructions stored on a computer-readable storage medium to perform certain steps or operations. Thus, embodiments of the present disclosure may also take the form of an entirely hardware embodiment, an entirely computer program product embodiment, and/or an embodiment that comprises combination of computer program products and hardware performing certain steps or operations.

Embodiments of the present disclosure are described below with reference to block diagrams and flowchart illustrations. Thus, it should be understood that each block of the block diagrams and flowchart illustrations may be implemented in the form of a computer program product, an entirely hardware embodiment, a combination of hardware and computer program products, and/or apparatuses, systems, computing devices, computing entities, and/or the like carrying out instructions, operations, steps, and similar words used interchangeably (e.g., the executable instructions, instructions for execution, program code, and/or the like) on a computer-readable storage medium for execution. For example, retrieval, loading, and execution of code may be performed sequentially such that one instruction is retrieved, loaded, and executed at a time. In some example embodiments, retrieval, loading, and/or execution may be performed in parallel such that multiple instructions are retrieved, loaded, and/or executed together. Thus, such embodiments can produce specifically configured machines performing the steps or operations specified in the block diagrams and flowchart illustrations. Accordingly, the block diagrams and flowchart illustrations support various combinations of embodiments for performing the specified instructions, operations, or steps.

provides an example overview of a systemthat can be used to practice embodiments of the present disclosure. The systemincludes a systemcomprising a computing entity. The systemmay communicate with one or more external computing entitiesA-N using one or more communication networks. Examples of communication networks include any wired or wireless communication network including, for example, a wired or wireless local area network (LAN), personal area network (PAN), metropolitan area network (MAN), wide area network (WAN), or the like, as well as any hardware, software and/or firmware required to implement it (e.g., network routers, and/or the like).

The systemincludes a storage subsystemconfigured to store at least a portion of the data utilized by the system. The computing entitymay be in communication with the external computing entitiesA-N.

The storage subsystemmay be configured to store the model definition data store and the training data store for one or more machine learning models. The computing entitymay be configured to receive requests and/or data from at least one of the external computing entitiesA-N, process the requests and/or data to generate outputs, and provide the outputs to at least one of the external computing entitiesA-N. In some embodiments, the external computing entityA, for example, may periodically update/provide raw and/or processed input data to the system. The external computing entitiesA-N may further generate user interface data (e.g., one or more data objects) corresponding to the outputs and may provide (e.g., transmit, send, and/or the like) the user interface data corresponding with the outputs for presentation to the external computing entityA (e.g., to an end-user).

The storage subsystemmay be configured to store at least a portion of the data utilized by the computing entityto perform one or more steps/operations and/or tasks described herein. The storage subsystemmay be configured to store at least a portion of operational data and/or operational configuration data including operational instructions and parameters utilized by the computing entityto perform the one or more steps/operations described herein. The storage subsystemmay include one or more storage units, such as multiple distributed storage units that are connected through a computer network. Each storage unit in the storage subsystemmay store at least one of one or more data assets and/or one or more data about the computed properties of one or more data assets. Moreover, each storage unit in the storage subsystemmay include one or more non-volatile storage or memory media including but not limited to hard disks, ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like.

The computing entitycan include an analysis engine and/or a training engine. The analysis engine may be configured to perform one or more data analysis techniques. The training engine may be configured to train the analysis engine in accordance with the data store stored in the storage subsystem.

provides an example computing entityin accordance with some embodiments discussed herein. In general, the terms computing entity, computer, entity, device, system, and/or similar words used herein interchangeably may refer to, for example, one or more computers, computing entities, desktops, mobile phones, tablets, notebooks, laptops, distributed systems, kiosks, input terminals, servers or server networks, blades, gateways, switches, processing devices, processing entities, set-top boxes, relays, routers, network access points, base stations, the like, and/or any combination of devices or entities adapted to perform the functions, steps/operations, and/or processes described herein. Such functions, steps/operations, and/or processes may include, for example, transmitting, receiving, operating on, processing, displaying, storing, determining, creating/generating, monitoring, evaluating, comparing, and/or similar terms used herein interchangeably. In one embodiment, these functions, steps/operations, and/or processes can be performed on data, content, information, and/or similar terms used herein interchangeably.

The computing entitymay include a network interfacefor communicating with various computing entities, such as by communicating data, content, information, and/or similar terms used herein interchangeably that can be transmitted, received, operated on, processed, displayed, stored, and/or the like.

In one embodiment, the computing entitymay include or be in communication with a processing element(also referred to as processors, processing circuitry, and/or similar terms used herein interchangeably) that communicate with other elements within the computing entityvia a bus, for example. As will be understood, the processing elementmay be embodied in a number of different ways including, for example, as at least one processor/processing apparatus, one or more processors/processing apparatuses, and/or the like.

For example, the processing elementmay be embodied as one or more complex programmable logic devices (CPLDs), microprocessors, multi-core processors, coprocessing entities, application-specific instruction-set processors (ASIPs), microcontrollers, and/or controllers. Further, the processing elementmay be embodied as one or more other processing devices or circuitry. The term circuitry may refer to an entirely hardware embodiment or a combination of hardware and computer program products. Thus, the processing elementmay be embodied as integrated circuits, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), hardware accelerators, other circuitry, and/or the like.

As will therefore be understood, the processing elementmay be configured for a particular use or configured to execute instructions stored in one or more memory elements including, for example, one or more volatile memoriesand/or non-volatile memories. As such, whether configured by hardware or computer program products, or by a combination thereof, the processing elementmay be capable of performing steps or operations according to embodiments of the present disclosure when configured accordingly. The processing element, for example in combination with the one or more volatile memoriesand/or or non-volatile memories, may be capable of implementing one or more computer-implemented methods described herein. In some implementations, the computing entitycan include a computing apparatus, the processing elementcan include at least one processor of the computing apparatus, and the one or more volatile memoriesand/or non-volatile memoriescan include at least one memory including program code. The at least one memory and the program code can be configured to, upon execution by the at least one processor, cause the computing apparatus to perform one or more steps/operations described herein.

The non-volatile memories(also referred to as non-volatile storage, memory, memory storage, memory circuitry, media, and/or similar terms used herein interchangeably) may include at least one non-volatile memory device, including but not limited to hard disks, ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like.

As will be recognized, the non-volatile memoriesmay store databases, database instances, database management systems, data, applications, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like. The term database, database instance, database management system, and/or similar terms used herein interchangeably may refer to a collection of records or data that is stored in a computer-readable storage medium using one or more database models, such as a hierarchical database model, network model, relational model, entity-relationship model, object model, document model, semantic model, graph model, and/or the like.

The one or more volatile memories(also referred to as volatile storage, memory, memory storage, memory circuitry, media, and/or similar terms used herein interchangeably) can include at least one volatile memory device, including but not limited to RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2SDRAM, DDR3SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, and/or the like.