Data Pagination Automation Method and Related Computer System

The present invention relates to a data pagination automation method and a related computer system, and more particularly, to a data pagination automation method and a related computer system capable of automatically paginating massive data.

With the importance of information safety, the application programming interface (API) of the conventional database provides the user to download data to reduce the risk of data leakage. However, the conventional application program interface performs the filtering after downloading the complete data. When downloading excessive data, the burden of the database and the transmission time is increased.

In order to solve the problem of massive data, pagination is utilized by the conventional application program interface, e.g. offset-based pagination, time-based pagination and cursor-based pagination. However, additional sorting logic is required for the offset-based pagination and the cursor-based pagination, which exponentially increases the quantity of the data sheet. The time-based pagination is easily affected by the volume of the data.

Therefore, improvements are necessary to the conventional techniques.

Therefore, the present invention provides a data pagination automation method and a related computer system to automatically paginate the massive data to reduce the service cost of the application program interface.

An embodiment of the present invention discloses a data pagination automation method, for processing a plurality of data items, comprises determining a sorting basis for sorting the plurality of data items according to an ordered unique key of the plurality of data items; sorting and paginating the plurality of data items according to the sorting basis; and updating a pagination information table of the plurality of data items and collecting corresponding data according to the ordered unique key and at least a filtering condition; wherein the step of sorting and paginating the plurality of data items according to the sorting basis comprises sorting the plurality of data items according to the ordered unique key, a pagination number and a quantity of data items of each pagination; reading data of each pagination according to the quantity of data items of each pagination and a cursor value; and storing data of each pagination and a last pagination according to the cursor value; wherein the pagination information table is updated with to a partial dynamic update method.

Another embodiment of the present invention discloses a computer system, for a server, comprises an application program interface, configured to provide a service interface; an arithmetic logic unit, configured to determine a sorting basis for sorting the plurality of data items according to an ordered unique key of the plurality of data items, and to sort and paginate the plurality of data items according to the sorting basis; and a backend module, configured to update a pagination information table of the plurality of data items and collect corresponding data according to the ordered unique key and at least a filtering condition; wherein the arithmetic logic unit is configured to determine a pagination number and a quantity of data items of each pagination, to sort the plurality of data items according to the ordered unique key, the pagination number and the quantity of data items of each pagination; to read data of each pagination according to the quantity of data items of each pagination and a cursor value; and to store data of each pagination and a last pagination according to the cursor value; wherein the pagination information table is updated with to a partial dynamic update method.

These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

1 FIG. 10 10 102 104 10 102 104 Please refer to, which is a schematic diagram of a computer systemaccording to an embodiment of the present invention. The computer systemincludes an application program interface API, an arithmetic logic unitand a backend module. The computer systemmay be utilized for a server of a data center or a database to store massive data. The application program interface API is configured to provide a service interface. For example, the application program interface API may provide an inquiry interface, such that after users or those who need the data input at least a filtering condition or inquiry condition, the results conforming to the filtering condition or the inquiry condition may be presented. The arithmetic logic unitis configured to determine a sorting basis for sorting according to an ordered unique key of the data stored in the server or the database, and to sort and paginate the data according to the sorting basis. The backend moduleis configured to update a pagination information table of the data and to collect corresponding data according to the ordered unique key and the filtering condition.

10 Therefore, the computer systemaccording to an embodiment of the present invention may rapidly perform the pagination for massive data with consistent logic to automatically paginate the data sheets, such that the service cost of the application program interface API is reduced.

2 FIG. 20 20 10 202 Step: Start; 204 Step: Determine the sorting basis for sorting the data according to the ordered unique key of the data; 206 Step: Sort and paginate the plurality of data items according to the sorting basis; 208 Step: Update the pagination information table of the data and collect corresponding data according to the ordered unique key and the filtering condition; 210 Step: End. Please refer to, which is a schematic diagram of a data pagination automation methodaccording to an embodiment of the present invention. The data pagination automation methodmay be utilized for processing the data of the computer systemand include the following steps:

10 In detail, the computer systemmay include a data sheet with all pagination information, wherein the data sheet may include different columns, e.g. time, identification (ID), name, class. Results of pagination may be different based on different columns for sorting. In another embodiment, results of pagination may be different based on different sorting order with identical column combination.

20 204 The data pagination automation methodaccording to an embodiment of the present invention determines the sorting basis for sorting in step. Since the data of data sheet are non-repetitive, i.e. a combination of all columns is a unique key, e.g. a data sheet of identification column with serial numbers. Notably, the unique key of a data sheet may be more than one.

After determining the column of the unique key as the sorting basis, the unique key is an ordered unique key (OUK). In this situation, only one result is generated for sorting according to the data sheet and the ordered unique key.

In an embodiment, the ordered unique key includes a scoring measurement Score (OUK). An optimal ordered unique key is the ordered unique key with the highest scoring measurement.

The scoring measurement Score (OUK) determines characteristics of each column of the data sheet, e.g. time, string or numerical value. Then, a uniqueness ratio and an empty ratio of each column are calculated, wherein the uniqueness ratio=number of uniqueness/number of all data items; the empty ratio=number of empty items/number of all data items.

The scoring measurement

wherein a column

In an embodiment, the weightings of column characteristic of time, string and numerical value are respectively 0.6, 0.3 and 0.1. In this example, the score measurement mainly sorts the time column. In another embodiment, the column score is higher when a repeatability of the column is higher and the empty value is less, i.e. the score measurement preferably selects and sorts the column with higher repeatability and less empty value. As can be known by formula (1), the scoring measurement Score (OUK) is the weighting and is exponentially decreased with the column score.

Therefore, in an embodiment, all candidate unique keys are listed, and then all candidate unique keys are expanded as candidate ordered unique keys. The scoring measurement Score (OUK) is utilized for scoring all candidate ordered unique keys to determine the optimal ordered unique key of the data sheet.

102 10 In addition to the above method for determining the candidate unique keys, the arithmetic logic unitof the computer systemaccording to an embodiment of the present invention may directly assign the candidate unique key. For example, in order to prevent repeat values from contaminating the data sheet and to maintain the characteristic of the original data sheet, a primary key is established to determine the sorting method. Alternatively, the candidate unique key may be selected to cooperate with the column characteristic of columns of the data sheet, such that the optimal ordered unique key, which conforms to the characteristic of the data sheet, may be selected.

102 10 In an embodiment, the ordered unique key with the highest scoring measurement Score (OUK) may be more than one. In this situation, the arithmetic logic unitof the computer systemaccording to an embodiment of the present invention may add a mechanism to select the only one optimal ordered unique key. For example, the optimal ordered unique key with the fewest column number, the optimal ordered unique key with the time column in the front of the data sheet, the optimal ordered unique key with the most time columns, the optimal ordered unique key with the most time columns or the optimal ordered unique key with the fewest value columns may be selected.

204 102 10 The unique sorting basis for the data sheet is determined according to the optimal ordered unique key determined in step. Then, the arithmetic logic unitof the computer systemmay sort and paginate the data according to the sorting basis and store related information of the pagination.

10 In detail, the computer systemmay include the pagination information table and a calculation method of hashed value, and store the pagination information according to related parameters of the pagination, e.g. the pagination number and the data items of each pagination. The pagination information table may include columns of the table name, the ordered unique key, the pagination page, the initial data value, the quantity of data items, the data hashed value and the update time.

3 FIG. 30 30 302 Step: Start; 304 Step: Determine a number of pagination and the number of data items of each pagination; 306 Step: Input the unique key; 308 Step: Sort the data according to the optimal ordered unique key; 310 Step: Set an initial page number as “−1” and a cursor value as empty; 312 Step: The page number “+1”; 314 Step: Obtain the data of each pagination, excluding the cursor; 316 Step: Store the pagination information; 318 326 320 Step: Determine whether the obtained data number is smaller than the number of data item of each pagination or not, if yes, go to step, if not, go to step; 320 Step: Set the cursor value as the last data item of the obtained data; 322 312 324 Step: Determine whether the page number is smaller than the pagination number “−1” or not, if yes, go to step, if not go to step; 324 Step: Store the information of the last pagination; 326 328 330 Step: Determine whether the number of the data item of the last pagination is 0 or not, if yes, go to step, if not go to step; 328 Step: Delete the last data page; 330 Step: End. Please refer to, which is a schematic diagram of a pagination flowchartof data sheets according to an embodiment of the present invention. The pagination flowchartof data sheets includes the following steps:

304 Notably, the data hashed value is obtained after the data value of the last item of data is deleted and encrypted with the hashed value. In addition, the unique key of stepis not limited to the optimal ordered unique key; when the data hashed value of the last pagination is not empty, the pagination of the data sheets is finished. When the data hashed value of the last pagination is empty, more data of the data sheet are left to be paginated.

4 FIG. 40 40 402 Step: Start; 404 Step: Input the ordered unique key and the filtering condition; 406 Step: Return all pagination page numbers conforming to the filtering condition; 408 Step: Obtain corresponding data hashed value according to the pagination page numbers; 410 Step: Recover the data hashed values according to a corresponding encryption method; 412 Step: Collect the recovered data; 414 Step: Return the data. In an embodiment, the application program interface API according to an embodiment of the present invention provides the interface for the users to set the inquiry conditions to obtain the data. Please refer to, which is a schematic diagram of a flowchartof obtaining data according to an embodiment of the present invention. The flowchartof obtaining data includes the following steps:

10 Notably, the query commands of the application program interface API of the computer system, e.g. between, larger and smaller, are the executable query commands, and not limited thereto.

20 206 The data pagination automation methodupdates the information of the pagination information table and collects corresponding data according to the ordered unique key and the filtering condition in step.

10 1. When the data needed by the user is at the last pagination, and the data hashed value of the last pagination is empty; 2. The data of the data sheet is moved because of scheduling; and 3. When too much pagination data, the pagination is split. In order to ensure that the user may obtain correct information of pagination, the pagination information table of the computer systemis updated under different situations.

10 In order to avoid the increase of the pagination cost, the computer systemaccording to an embodiment of the present invention partially paginates the pagination information table to determine whether the data needed by the user is at the pagination with empty hashed value or not. If yes, no splitting is performed; if not, data splitting is performed until the requirement of the user is satisfied to avoid unnecessary splitting cost.

5 FIG. 50 50 502 Step: Start; 504 Step: Input the ordered unique key and the filtering condition; 506 508 518 Step: Determine whether the initial data value of the last pagination conforms to the filtering condition or not, if yes, go to step, if not, go to step; 508 Step: Set the last item of data of a second-to-last page as the cursor value, set the value of the initial page number as a second-to-last page number; 510 312 330 322 Step: Go to steps-, wherein the condition of stepis replaced with “determine whether the last item of data conforms to the filtering condition or not”; 512 Step: Recover the data hashed value according to the corresponding encryption method; 514 Step: Collect the recovered data; 516 Step: Return the data; 518 Step: End. Please refer to, which is a schematic diagram of a flowchartof updating pagination information table according to an embodiment of the present invention. The flowchartof updating pagination information table includes the following steps:

10 60 60 6 FIG. 602 Step: Start; 604 Step: Input the time information or a frequency of the data sheet for update; 606 Step: Receive a notification of the pagination information from the user; 608 614 610 Step: Determine whether the update time of the pagination page number of the data sheet is later than the latest one or not, if yes, go to step, if not, go to step; 610 Step: Obtain the data between the initial data value of the pagination page number and the initial data value of a following pagination page number; 612 Step: Update the information of the pagination page number; 614 Step: Obtain the data hashed value corresponding to the pagination page number; 616 Step: Recover the data hashed value according to the encryption method; 618 Step: End. On the other hand, the computer systemaccording to an embodiment of the present invention may update the data sheet with a fixed periodic scheduling method, e.g. the data sheet is updated every 19:00. Please refer to, which is a schematic diagram of a flowchartof updating pagination information table according to an embodiment of the present invention. The flowchartof updating pagination information table includes the following steps:

10 Therefore, the computer systemaccording to an embodiment of the present invention may partially and dynamically update the pagination information table. After receiving the notification from the user, the latest state of the pagination may be confirmed, if yes, the current pagination information is directly transmitted to the user; if not, the pagination information is updated and transmitted to the user.

10 70 70 7 FIG. 702 Step: Start; 704 Step: Set a maximal data items of each pagination; 706 Step: Obtain the pagination page number and record a main pagination page number; 708 Step: Recover the original data hashed value according to the main pagination page number and the encryption method; 710 30 Step: Perform the pagination flowchartof data sheets, wherein the definition of page number is replaced with “the main pagination page number and the page number”; 712 Step: Take the main pagination page number as a first sorting basis, take the page number as a second sorting basis, and sort the pagination information accordingly; 714 Step: Rename the page number and start with the page number 716 Step: End. Since the download speed of pagination might be affected after the data sheets are updated with excessive data and additional pagination. In order to avoid such situation, the computer systemaccording to an embodiment of the present invention may manage the data items of each pagination and perform the splitting for the data sheet. Please refer to, which is a schematic diagram of a splitting flowchartaccording to an embodiment of the present invention. The splitting flowchartincludes the following steps:

10 70 10 104 The computer systemaccording to an embodiment of the present invention may dynamically update the pagination when performing the splitting flowchart, wherein the variation is performed when the user requests a demand of obtaining the data of a pagination, and the computer systemdetermines whether the maximal data number of the pagination is met or not, if yes, the backend modulesplits the pagination information table, and replaces the pagination information table with the split pagination information table.

In addition, the original pagination information table may be utilized for providing the data during the split period and the service quality for the user would not be affected.

Notably, those skilled in the art may make proper modifications to the data pagination automation method according to different requirements, not limited thereto and can be modified according to different user's preferences or system settings, which are all within the scope of the present invention.

In summary, the present invention provides a data pagination automation method and a related computer system, which is capable of rapidly paginating massive data with consistent logic and achieves the data pagination automation, so as to reduce service cost of the application program interface.

Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.