A system and methodology for data processing combining stream processing and spreadsheet computation comprises: one or more Cloudscale Stream Processors (CSPs), a Cloudscale-enabled spreadsheet (including a plug-in module), a web services architecture, and one or more data sources. Each CSP takes a set of streams of input data and outputs a set of streams of data. Each stream is a time-ordered stream of files which when concatenated together comprise the stream. The Cloudscale-enabled spreadsheet is includes a plug in module that operates in conjunction with a spreadsheet program and one or more stream processors. The Cloudscale-enabled spreadsheet provides tools and technologies for user authentication, stream discovery, data source extraction and new stream design. It includes features for design of new stream processors, as well as features for checking and validation, cycle and synchronization controls for stream processors and spreadsheets.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A system for processing of data streams, the system comprising: at least one computer system including at least a processor and memory; at least one stream processing module that receives one or more input data streams and produces one or more output data streams, wherein each of the input data streams and output data streams are a time-ordered group of files concatenated together; a spreadsheet module that repeatedly performs a two-step cycle in which during the first step, the output data streams produced by said at least one stream processing module are entered into particular spreadsheet cells of a spreadsheet, and then in the second step, the spreadsheet module calculates values in the spreadsheet based on the output data streams entered into particular spreadsheet cells of the spreadsheet during the first step of the cycle; and a plug-in module to said spreadsheet module enabling a user to: design said at least one stream processing module utilizing the spreadsheet module, and select data for processing by said at least one stream processing module, wherein the plug-in module specifies rows and columns of the output data streams produced by said at least one stream processing module, enters the output data streams into particular spreadsheet cells of the spreadsheet, and performs calculations by the spreadsheet module on the output data streams entered into the particular spreadsheet cells of the spreadsheet, wherein said at least one stream processing module is external to the spreadsheet and the plug-in module.
2. The system of claim 1 , wherein said at least one stream processing module performs static and dynamic parallel processing of said one or more input streams of data.
3. The system of claim 1 , wherein said at least one stream processing module performs selected ones of time-based and punctuation-based window processing of said one or more input streams of data.
4. The system of claim 1 , wherein said at least one stream processing module is designed from a set of building blocks in which each building block is associated with one or more spreadsheet cells of the spreadsheet.
5. The system of claim 4 , wherein said set of building blocks can be constructed in any order.
6. The system of claim 4 , wherein said at least one stream processing module is designed without requiring any ordering on the building blocks, or any other additional structuring or control flow.
7. The system of claim 4 , wherein a building block comprises an input block corresponding to an input column of an input stream of data.
8. The system of claim 4 , wherein a building block comprises an assignment block corresponding to a column of an input stream of data defined by a formula.
9. The system of claim 8 , wherein said assignment block has an optional parameter specifying parallel processing on groups.
10. The system of claim 8 , wherein each assignment block has an optional parameter specifying window processing.
11. The system of claim 1 , wherein said at least one stream processing module is controlled through user interaction with the spreadsheet module, including controls for starting and stopping processing of one or more input streams of data by said at least one stream processing module.
12. The system of claim 11 , wherein the user may specify a time period for performance of said two-step cycle using the spreadsheet module.
13. The system of claim 12 , wherein the time period for performance of the two-step cycle can be specified using selected ones of manual start controls, setting a specific time period for performance of the cycle, and setting the cycle to be performed as fast as possible.
14. The system of claim 1 , wherein said plug-in module enables a user to select a first subset of one or more of said at least one stream processors for which processing of input streams of data is to be stopped and later resumed, while specifying another subset of one or more of said at least one stream processing modules that are to continuing to process input streams of data while the first subset is inactive.
15. The system of claim 1 , wherein said at least one stream processing system processes said one or more input streams of data for a period of time specified by user input using the spreadsheet module.
16. The system of claim 15 , wherein the period of time for during which said at least one stream processing module continues to process input streams of data is governed by a finite-term lease that may be renewed, reduced or extended by the user at any time using the spreadsheet module.
17. The system of claim 1 , wherein said one or more input streams of data to said at least one stream processing module comprise live data streams and are processed by said at least one stream processing module in real-time.
18. The system of claim 1 , wherein said one or more input streams of data to said at least one stream processing module is of a plurality of forms including selected ones of live time-ordered data streams, stored time-ordered data streams, and unordered tables of data.
19. The system of claim 1 , wherein said at least one stream processing module includes one or more stream processing modules provided as cloud computing services, wherein said one or more input streams of data are processed by said one or more stream processing modules remotely from the spreadsheet module by the cloud computing services.
20. The system of claim 19 , wherein processing of input streams of data is carried out by a plurality of stream processing modules provided as cloud computing services on a plurality of distinct cloud computing architectures.
21. The system of claim 20 , wherein said plurality of distinct cloud computing architectures include public external cloud architectures and private internal cloud architectures.
22. The system of claim 1 , wherein the spreadsheet module graphically displays results of calculating output data produced by said at least one stream processing module as input into particular spreadsheet cells of the spreadsheet module.
23. The system of claim 1 , wherein the spreadsheet module provides connectivity to other systems and users for collaborative data processing.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 9, 2010
July 22, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.