A query that is frequently processed to access an object storage is identified. Results from the query returned from the object storage is transformed into a relational database format as a materialized view. When the query is submitted a subsequent time, updated results are managed from the materialized view, other materialized views, and/or the object storage when needed.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
2. The method of claim 1, wherein transforming further includes processing a Structured Query Language (SQL) SELECT Data Manipulation Language (DML) statement that defines a mapping between the storage object values to the relational database format.
3. The method of claim 1 wherein maintaining further includes, maintaining a filter representing text-formatted query conditions for the query with the materialized view.
4. The method of claim 3, wherein maintaining further includes periodically submitting the filter for the query to the object storage and updating the materialized view with new modification timestamps and new storage object values.
5. The method of claim 4, wherein updating further includes removing at least one storage object identifier and corresponding storage object values associated with the at least one storage object identifier.
6. The method of claim 4, wherein updating further includes adding at least one storage object identifier and corresponding storage object values associated with the at least one storage object identifier.
7. The method of claim 1, wherein providing further includes updating a portion of the storage object values with data values obtained from a different materialized view and a corresponding modification timestamp when a particular storage object identifier of the different materialized view matches a corresponding storage object identifier of the materialized view and has a more recent modification timestamp.
8. The method of claim 1, wherein providing further includes updating a portion of the storage object values from the object storage based on a filter associated with a particular storage object identifier included in the materialized view.
9. The method of claim 1, wherein providing further includes provide the materialized view to satisfy the query without accessing the object storage.
This invention relates to database systems and specifically to optimizing query performance by using materialized views to avoid accessing object storage. The problem addressed is the inefficiency of repeatedly accessing object storage for frequently queried data, which can slow down database operations and increase resource consumption. The solution involves providing a materialized view that stores precomputed query results, allowing queries to be satisfied directly from the materialized view without needing to access the underlying object storage. This reduces latency and improves performance by eliminating the need for repeated storage access operations. The materialized view is dynamically updated to ensure it reflects the latest data, maintaining accuracy while still providing the benefits of faster query execution. The approach is particularly useful in systems where object storage access is a bottleneck, such as large-scale data warehouses or distributed database environments. By leveraging the materialized view, the system can handle high query loads more efficiently, reducing both computational overhead and storage access delays. The invention ensures that queries are resolved quickly by serving results from the materialized view, while still maintaining data consistency through periodic updates or triggers that refresh the view when the underlying data changes. This method is applicable to various database architectures, including those that rely on cloud-based or distributed storage systems.
11. The method of claim 10 further comprising, periodically refreshing the materialized view from the object storage based on a refresh policy.
13. The method of claim 12, wherein managing further includes using the storage object identifiers and the modification timestamps to determine where different portions of the materialized view are to be updated from.
14. The method of claim 13, wherein using further includes obtaining some of the updated results from different materialized views.
15. The method of claim 14, wherein obtaining further includes obtaining another portion of the updated results from the object storage based on filters associated with particular storage object identifiers in the materialized view.
A system and method for efficiently retrieving and filtering data from an object storage system using materialized views. The technology addresses the challenge of optimizing query performance in distributed storage environments where data is stored as objects, often requiring complex filtering operations that can be computationally expensive. The method involves maintaining a materialized view that stores precomputed results of queries, including metadata and identifiers for storage objects. When a query is executed, the system retrieves a portion of the updated results from the object storage based on filters associated with specific storage object identifiers in the materialized view. This approach reduces the need for repeated full scans of the object storage, improving query efficiency by leveraging precomputed data and selective retrieval of relevant objects. The system dynamically updates the materialized view to ensure the results remain current, balancing performance with data consistency. The method is particularly useful in large-scale data processing environments where minimizing latency and resource usage is critical.
16. The method of claim 12 further comprising, periodically refreshing the materialized view by obtaining new results from the object storage for the query based on a refresh policy.
17. The method of claim 12 further comprising, automatically obtaining new results from the object storage for the query when a determination is made that more than a predefined portion of the results are out of date with the object storage or other materialized views.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
December 28, 2018
October 11, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.