Terminal and server for interactions between a streamer and a user responsive to a gift

This application is a Continuation application of U.S. Ser. No. 18/306,052, filed on Apr. 24, 2023, which is based on and claims the benefit of priority from Japanese Patent Applications Serial Nos. 2022-086444 (filed on May 26, 2022), and 2022-125308 (filed on Aug. 5, 2022), the contents of each of which are hereby incorporated by reference in their entirety.

The present disclosure relates to a terminal and a server.

With the development of IT technology, the way information is exchanged has changed. In the Showa period (1926-1989), one-way information communication via newspapers and television was the main stream. In the Heisei period (1990-2019), with the widespread availability of cell phones and personal computers, and the significant improvement in Internet communication speed, instantaneous interactive communication services such as chat services emerged, and on-demand video streaming services also become popular as storage costs were reduced. And nowadays or in the Reiwa period (2019 to present), with the sophistication of smartphones and further improvements in network speed as typified by 5G, services that enable real-time communication through video, especially live-streaming services, are gaining recognition. The number of users of live-streaming services is expanding, especially among young people, as such services allow people to share the same good time even when they are in the separate locations from each other.

Gifts, such as tips (Nagesen), play a key role in making live-streams exciting and constitute an important source of income for live-streamers. Japanese Patent Application Publication No. 2020-017870 describes a technique to make a live-stream more exciting by showing a Nagesen (tipping) effect with which a donation from a viewer to a live-streamer is shown on the live-stream screen.

In general, if a viewer gives a gift to a streamer within a live-stream, the streamer can know who gives what real time by referring to an effect and a comment. Thus, the live-streamer can immediately thank the viewer, show performance or respond in any other ways. Such responses can be witnessed by everyone participating in the live-stream. This has been established as one of the ways of communication live-streams provide for between streamers and viewers.

If a viewer gives a gift to a streamer outside a live-stream, on the other hand, the streamer gives no response to the viewer, or, if any, only posts an automatic comment on the timeline. Such responses can hardly satisfy the viewer who has given the gift.

In view of the above, one object of the disclosure is to provide a technology that can bring about interactions between a streamer and a user who gives a gift to the streamer outside a live-stream.

One aspect of the disclosure relates to a terminal of a user. The terminal includes one or more processors; and memory storing one or more computer programs configured to be executed by the one or more processors. The one or more computer programs include instructions for: receiving, from the user, an instruction to use a gift for a streamer while the user is not participating in a live-stream of the streamer; and causing an output unit to output an effect corresponding to the use of the gift by the user while the streamer is live-streaming.

Another aspect of the disclosure also relates to a terminal of a streamer of a live-stream. The terminal includes: one or more processors; and memory storing one or more computer programs configured to be executed by the one or more processors. The one or more computer programs include instructions for: showing an object on a display during the live-stream of the streamer, the object corresponding to use of a gift for the streamer by a user while the user is not participating in the live-stream of the streamer.

A further another aspect of the present disclosure relates to a server. The server includes: a holding unit for holding information regarding a gift used by a user for a streamer while the user is not participating in a live-stream of the streamer; a receiving unit for receiving, from a terminal of the user over a network, a request made by the user to join the live-stream of the streamer; and a determining unit for, upon reception of the request, referring to the holding unit to determine whether to have an effect output, the effect corresponding to the use of the gift by the user.

It should be noted that the components described throughout this disclosure may be interchanged or combined. The components, features, and expressions described above may be replaced by devices, methods, systems, computer programs, recording media containing computer programs, etc. Any such modifications are intended to be included within the spirit and scope of the present disclosure.

Like elements, components, processes, and signals throughout the figures are labeled with same or similar designations and numbering, and the description for the like elements will not be hereunder repeated. For purposes of clarity and brevity, some of the components that are less related and thus not described are not shown in the figures.

A live-streaming system relating to an embodiment of the disclosure allows users to give gifts to other users without participating in live-streams. For example, users can send gifts to their favorite streamers even while the streamers are not live-streaming. Hereinafter, the term “in-stream gift” refers to a gift given by a user to a streamer within a live-stream of the streamer or while the user is participating in the live-stream, and the term “out-of-stream gift” denotes a gift given by a user to a streamer outside a live-stream of the streamer or while the user is not participating in the live-stream. In other words, out-of-stream gifts can be any gifts other than in-stream gifts.

At any timing after a user gives an out-of-stream gift to a streamer, the streamer may start live-streaming. The user who has given the out-of-stream gift may also join the live-stream. In the live streaming system relating to the embodiment, when the user who has given the out-of-stream gift joins the live-stream, an effect indicating the usage of the out-of-stream gift appears on the screen of the live-stream. This can make the user who has given the out-of-stream gift feel special and tell the streamer that the user who has given the out-of-stream gift to the streamer has participated in the live-stream. The streamer can thus thank the user for the out-of-stream gift again within the live-stream. This in turn encourages users to use out-of-stream gifts, strengthening the connection between users and streamers through live-streams.

1 FIG. 1 FIG. 1 1 1 2 1 10 20 30 30 30 a b schematically illustrates the configuration of a live-streaming systemaccording one embodiment of the disclosure. The live-streaming systemprovides an interactive live-streaming service that allows a streamer LV (also referred to as a liver or distributor) and a viewer AU (also referred to as audience) (AU, AU, . . . ) to communicate in real time. As shown in, the live-streaming systemincludes a server, a user terminalon the streamer side, and user terminals(,, . . . ) on the audience side. The streamer and viewers may be collectively referred to as users. In addition to the streamer who is live-streaming and the viewers who watch the live-stream, there may be users who have logged in the live-streaming platform but is neither live-streaming nor watching the live-stream. Such users are referred to as active users. As used herein, users who receive out-of-stream gifts may also be referred to as streamers. This is because such users live-stream all the time and are thus expected to live-stream at any timing after receiving out-of-stream gifts.

10 20 30 10 20 30 The servermay be one or more information processing devices connected to a network NW. The user terminalsandmay be, for example, mobile terminal devices such as smartphones, tablets, laptop PCs, recorders, portable gaming devices, and wearable devices, or may be stationary devices such as desktop Pcs. The server, the user terminal, and the user terminalsare interconnected so as to be able to communicate with each other over the various wired or wireless network NW.

1 10 20 1 10 30 30 The live-streaming systeminvolves the streamer LV, the viewers AU, and an administrator (not shown) who manages the server. The streamer LV is a person who broadcasts contents in real time by recording the contents with his/her user terminaland uploading them directly to the server. Examples of the contents may include the streamer's own songs, talks, performances, fortune-telling, gameplays, and any other contents. The administrator provides a platform for live-streaming contents on the server, and also mediates or manages real-time interactions between the streamer LV and the viewers AU. The viewers AU access the platform at their user terminalsto select and view a desired content. During live-streaming of the selected content, the viewer AU performs operations to comment, cheer, or ask fortune-telling via the user terminal, the streamer LV who is delivering the content responds to such a comment, cheer, or request and such response is transmitted to the viewer AU via video and/or audio, thereby establishing an interactive communication.

20 30 As used herein, the term “live-streaming” or “live-stream” may mean a mode of data transmission that allows a content recorded at the user terminalof the streamer LV to be played and viewed at the user terminalsof the viewers AU substantially in real time, or it may mean a live broadcast realized by such a mode of transmission. The live-streaming may be achieved using existing live-streaming technologies such as HTTP Live Streaming, Common Media Application Format, Web Real-Time Communications, Real-Time Messaging Protocol and MPEG DASH. The live-streaming includes a transmission mode in which, while the streamer LV is recording contents, the viewers AU can view the contents with a certain delay. The delay is acceptable as long as interaction between the streamer LV and the viewers AU can be at least established. Note that the live-streaming is distinguished from so-called on-demand type transmission, in which contents are entirely recorded and the entire data is once stored on the server, and the server provides users with the data at any subsequent time upon request from the users.

20 30 20 30 20 30 10 10 The term “video data” herein refers to data that includes image data (also referred to as moving image data) generated using an image capturing function of the user terminalsandand audio data generated using an audio input function of the user terminalsand. Video data is played back on the user terminalsand, so that the users can view contents. In this embodiment, it is assumed that between video data generation at the streamer's user terminal and video data reproduction at the viewer's user terminal, processing is performed onto the video data to change its format, size, or specifications of the data, such as compression, decompression, encoding, decoding, or transcoding. However, such processing does not substantially change the content (e.g., video images and audios) represented by the video data, so that the video data after such processing is herein described as the same as the video data before such processing. In other words, when video data is generated at the streamer's user terminal and then played back at the viewer's user terminal via the server, the video data generated at the streamer's user terminal, the video data that passes through the server, and the video data received and reproduced at the viewer's user terminal are all the same video data.

1 FIG. 20 10 20 20 In the example in, the streamer LV is live-streaming his/her talk. The user terminalof the streamer LV generates video data by recording images and sounds of the streamer LV who is talking, and the generated data is transmitted to the serverover the network NW. At the same time, the user terminaldisplays the recorded video image VD of the streamer LV on the display of the user terminalto allow the streamer LV to check what is to be live-streamed.

30 30 1 2 1 2 1 2 30 30 20 30 30 20 a b a b a b The user terminalsandof the viewers AUand AUrespectively, who have requested the platform to enable them to view the live-stream of the streamer LV, receive video data related to the live-stream over the network NW and reproduce the received video data, to display video images VDand VDon the displays and output audio through the speakers. The video images VDand VDdisplayed at the user terminalsand, respectively, are substantially the same as the video image VD captured by the user terminalof the streamer LV, and the audio outputted at the user terminalsandis substantially the same as the audio recorded by the user terminalof the streamer LV.

20 30 30 1 2 1 30 10 20 30 30 1 2 30 30 1 2 1 1 a b a a b a b Recording of the images and sounds at the user terminalof the streamer LV and reproduction of the video data at the user terminalsandof the viewers AUand AUare performed substantially simultaneously. The viewer AUmay type a comment about the talk of the streamer LV on the user terminal, and the servermay display the comment on the user terminalof the streamer LV in real time and also display the comment on the user terminalsandof the viewers AUand AU, respectively. The streamer LV may read the comment and develop his/her talk to cover and respond the comment, and the video and sound of the talk are displayed on the user terminalsandof the viewers AUand AU, respectively. This interactive action is recognized as establishment of a conversation between the streamer LV and the viewer AU. In this way, the live-streaming systemrealizes the live-streaming that enables the interactive communication, not one-way communication.

2 FIG. 1 FIG. 2 FIG. 20 30 20 is a block diagram showing functions and configuration of the user terminalof. The user terminalshave the same functions and configuration as the user terminal. The blocks inand the subsequent block diagrams may be realized by elements such as a computer CPU or a mechanical device in terms of hardware, and can be realized by a computer program or the like in terms of software. The blocks shown in the drawings are, however, functional blocks realized by cooperative operation between hardware and software. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

20 30 20 30 20 30 20 30 10 20 30 20 30 20 30 10 20 30 The streamer LV and the viewers AU download and install a live-streaming application program (hereinafter referred to as a live-streaming application), onto the user terminalsandfrom a download site over the network NW. Alternatively, the live-streaming application may be pre-installed on the user terminalsand. When the live-streaming application is executed on the user terminalsand, the user terminalsandcommunicate with the serverover the network NW to implement various functions. Hereinafter, the functions implemented by (processors such as CPUs of) the user terminalsandby running the live-streaming application will be described as functions of the user terminalsand. These functions are realized in practice by the live-streaming application on the user terminalsand. In any other embodiments, these functions may be realized by a computer program that is written in a programming language such as HTML (HyperText Markup Language), transmitted from the serverto web browsers of the user terminalsandover the network NW, and executed by the web browsers.

20 100 10 200 10 400 100 200 400 100 200 400 400 308 10 The user terminalincludes a streaming unitfor recording the user's image and sound to generate and provide video data to the server, a viewing unitfor acquiring and reproduce the video data from the server, and an out-of-stream processing unitfor processing requests made by active users. The user activates the streaming unitto live-stream, the viewing unitto view a live-stream, and the out-of-stream processing unitto look for a live-stream, view a streamer's profile, or watch an archive. The user terminal having the streaming unitactivated is the streamer's terminal, i.e., the user terminal that generates video data, the user terminal having the viewing unitactivated is the viewer's terminal, i.e., the user terminal that reproduces video data, and the user terminal having the out-of-stream processing unitactivated is the active user's terminal. The out-of-stream processing unitworks cooperatively with the out-of-stream gift processing unitof serverto perform processing related to out-of-stream gifts.

100 102 104 106 108 110 102 102 104 104 106 102 104 10 106 102 104 106 2 FIG. 2 FIG. The streaming unitincludes an image capturing control unit, an audio control unit, a video transmission unit, a streamer-side UI control unit, and a streamer-side communication unit. The image capturing control unitis connected to a camera (not shown in) and controls image capturing performed by the camera. The image capturing control unitobtains image data from the camera. The audio control unitis connected to a microphone (not shown in) and controls audio input from the microphone. The audio control unitobtains audio data through the microphone. The video transmission unittransmits video data including the image data obtained by the image capturing control unitand the audio data obtained by the audio control unitto the serverover the network NW. The video data is transmitted by the video transmission unitin real time. That is, the generation of the video data by the image capturing control unitand the audio control unit, and the transmission of the generated video data by the video transmission unitare performed substantially at the same time.

108 108 106 108 108 10 108 2 FIG. 2 FIG. The streamer-side UI control unitcontrols an UI for the streamer. The streamer-side UI control unitis connected to a display (not shown in), and displays a video on the display by reproducing the video data that is to be transmitted by the video transmission unit. The streamer-side UI control unitis also connected to input means (not shown in) such as touch panels, keyboards, and displays, and obtains the streamer's input via the input means. The streamer-side UI control unitsuperimposes a predetermined frame image on the video image. The frame image includes various user interface objects (hereinafter simply referred to as “objects”) for accepting inputs from the streamer, comments entered by the viewers, effects corresponding to gifts, and information retrieved from the server. The streamer-side UI control unitaccepts, for example, the streamer's inputs made by the streamer tapping the objects.

110 10 110 108 10 110 10 The streamer-side communication unitcontrols communication with the serverduring a live-stream. The streamer-side communication unittransmits the content of the streamer's input that has been obtained by the streamer-side UI control unitto the serverover the network NW. The streamer-side communication unitreceives various information associated with the live-stream from the serverover the network NW.

200 202 204 204 10 204 10 The viewing unitincludes a viewer-side UI control unitand a viewer-side communication unit. The viewer-side communication unitcontrols communication with the serverduring a live-stream. The viewer-side communication unitreceives, from the serverover the network NW, video data related to the live-stream in which the streamer and the viewer participate.

202 202 202 202 10 10 204 202 10 2 FIG. 2 FIG. The viewer-side UI control unitcontrols the UI for the viewer. The viewer-side UI control unitis connected to a display and a speaker (not shown in), and reproduces the received video data so that video images are displayed on the display and sounds are output through the speaker. The state where the images and sounds are respectively output through the display and speaker can be referred to as “the video data is reproduced”. The viewer-side UI control unitis also connected to input means (not shown in) such as touch panels, keyboards, and displays, and obtains viewer's input via the input means. The viewer-side UI control unitsuperimposes a predetermined frame image on an image generated from the video data obtained from the server. The frame image includes various objects for accepting inputs from the viewer, comments entered by the viewer, effects corresponding to gifts, and information retrieved from the server. The viewer-side communication unittransmits the content of the viewer's input that has been obtained by the viewer-side UI control unitto the serverover the network NW.

400 402 404 402 402 402 402 The out-of-stream processing unitincludes an out-of-stream UI control unitand an out-of-stream communication unit. The out-of-stream UI control unitcontrols an UI for the active user. For example, the out-of-stream UI control unitgenerates a live-stream selection screen and shows the screen on the display. The live-stream selection screen presents a list of live-streams to which the active user is currently invited to participate to allow the active user to select a live stream. The out-of-stream UI control unitgenerates a profile screen for any user and shows the screen on the display. The out-of-stream UI control unitplays back an archived past live-stream, which is recorded and stored.

404 10 404 10 404 10 The out-of-stream communication unitcontrols communication with the serverthat takes place outside a live-stream. The out-of-stream communication unitreceives, from the serverover the network NW, information necessary to generate the live-stream selection screen, information necessary to generate the profile screen, and archived data. The out-of-stream communication unittransmits the content of the active user's input to the serverover the network NW.

3 FIG. 1 FIG. 10 10 302 304 306 308 310 322 324 326 328 314 318 320 330 332 334 is a block diagram showing functions and configuration of the serverof. The serverincludes a streaming information providing unit, a relay unit, an in-stream gift processing unit, an out-of-stream gift processing unit, a payment processing unit, a streaming start receiving unit, an out-of-stream gift totaling unit, an effect determining unit, a ranking unit, a stream DB, a user DB, a gift DB, an out-of-stream gift history DB, an out-of-stream gift ranking DB, and an out-of-stream gift total table.

4 FIG. 3 FIG. 314 314 314 1 is a data structure diagram showing an example of the stream DBof. The stream DBholds information regarding live-streams currently taking place. The stream DBstores a stream ID identifying a live-stream on a live streaming platform provided by the live-streaming system, a streamer ID, which is a user ID identifying the streamer who provides the live-stream, and a viewer ID, which is a user ID identifying a viewer of the live-stream, in association with each other.

1 In the live-streaming platform provided by the live-streaming systemof the embodiment, when a user live-streams, the user is referred to as a streamer, and when the same user views a live-stream distributed by another user, the user is referred to as a viewer. Therefore, the distinction between a streamer and a viewer is not fixed, and a user ID registered as a streamer ID at one time may be registered as a viewer ID at another time.

5 FIG. 3 FIG. 318 318 318 is a data structure diagram showing an example of the user DBof. The user DBholds information regarding users. The user DBstores a user ID identifying a user, points owned by the user, and the reward awarded to the user, in association with each other. The points are an electronic representation of value circulated in the live-streaming platform. The user can purchase the points using a credit card or other means of payment. The reward is an electronic representation of value defined in the live-streaming platform and is used to determine the amount of money the streamer receives from the administrator of the live-streaming platform. In the live-streaming platform, when a viewer gives a gift to a streamer within or outside a live-stream, the viewer's points are consumed and, at the same time, the streamer's reward is increased by a corresponding amount.

6 FIG. 3 FIG. 320 320 It can be purchased in exchange for the points or money, or can be given for free. It can be given to a streamer by a viewer within a live-stream and by an active user outside a live-stream. Giving a gift to a streamer is also referred to as using the gift or throwing the gift. Some gifts may be purchased and used at the same time, and some gifts may be used by the viewer at any time after purchased. When a gift is given to a streamer, a corresponding reward is awarded to the streamer. When a gift is used, the use may trigger an effect associated with the gift. For example, an effect corresponding to the gift will appear on the live-streaming room screen. is a data structure diagram showing an example of the gift DBof. The gift DBholds information regarding gifts available for the viewers in relation to the live-streaming. A gift is electronic data with the following characteristics:

320 The gift DBstores: a gift ID for identifying a gift; a reward to be awarded to a streamer when the gift is given to the streamer; price points that is the price to be paid for the use of the gift; and a type of the gift, in association with each other. A viewer is able to give a desired gift to a streamer by paying the price points of the desired gift while viewing a live-stream of the streamer or outside a live-stream of the streamer. The payment of the price points may be made by appropriate electronic payment means. For example, the payment may be made by the viewer paying the price points to the administrator. Alternatively, bank transfers or credit card payments may be available. The administrator is able to desirably set the relationship between the reward to be awarded and the price points. For example, it may be set as the reward to be awarded=the price points. Alternatively, points obtained by multiplying the reward to be awarded by a predetermined coefficient such as 1.2 may be set as the price points, or points obtained by adding predetermined fee points to the reward to be awarded may be set as the price points.

Gifts are classified into types “in-stream,” “out-of-stream,” and “both.” The type “in-stream” indicates an in-stream gift, “out-of-stream” an out-of-stream gift, and “both” that the gift can be used either within or outside a live-stream. In-stream gifts are configured unavailable outside a live-stream, and out-of-stream gifts are configured unavailable within a live-stream.

10 10 306 10 320 306 202 The user terminal of a viewer participating in a live-stream requests a list of available gifts from the servervia the network. Upon receipt of the request, the in-stream gift processing unitof the serverrefers to the gift DBand identifies gift IDs of the gifts classified into the “in-stream” and “both” types. The in-stream gift processing unitgenerates a list of the identified gift IDs and transmits the list to the user terminal of the requester. The viewer-side UI control unitin the user terminal of the requester shows on the display the gift icons of the gifts identified by the gift IDs in the received list.

10 10 308 10 320 308 402 The user terminal of an active user requests a list of available gifts from the servervia the network. Upon receipt of the request, the out-of-stream gift processing unitof the serverrefers to the gift DBand identifies gift IDs of the gifts classified into the “out-of-stream” and “both” types. The out-of-stream gift processing unitgenerates a list of the identified gift IDs and transmits the list to the user terminal of the requester. The out-of-stream UI control unitin the user terminal of the requester shows on the display the gift icons of the gifts identified by the gift IDs in the received list.

7 FIG. 3 FIG. 330 330 330 330 is a data structure diagram showing an example of the out-of-stream gift history DBof. The out-of-stream gift history DBholds information regarding out-of-stream gifts that have been used by users for streamers while the users are not participating in live-streams of the streamers. In particular, the out-of-stream gift history DBholds histories of out-of-stream gifts used. The out-of-stream gift history DBholds, for each out-of-stream gift used, the user ID of the recipient of the out-of-stream gift, the user ID of the giver of the out-of-stream gift, the time the out-of-stream gift was used, the gift ID of the out-of-stream gift, and the price points of the out-of-stream gift, in association with each other.

8 FIG. 3 FIG. 332 332 332 332 is a data structure diagram showing an example of the out-of-stream gift ranking DBof. The out-of-stream gift ranking DBholds ranking information that indicates rankings of the users who have given out-of-stream gifts to a specific streamer by the quantity of out-of-stream gifts. An out-of-stream gift ranking DBis generated for each streamer. The out-of-stream gift ranking DBholds a rank, the user ID of a user who has given out-of-stream gifts to a specific streamer, and the total of the price points of the out-of-stream gifts given by the user to the specific streamer, in association with each other.

9 FIG. 3 FIG. 334 334 334 334 is a data structure diagram showing an example of the out-of-stream gift total tableof. The out-of-stream gift total tableis generated upon a start of a live-stream by a streamer, in association with the live-stream. The out-of-stream gift total tableholds the result of totaling the out-of-stream gifts used for the streamer of the live-stream. The out-of-stream gift total tableholds the user ID of a user who has given one or more out-of-stream gifts to the streamer and the total of the price points of the out-of-stream gifts given by the user to the streamer, in association with each other.

The following now describes how the respective units work in two different phases: within a live-stream; and outside a live-stream.

(Outside Live-Stream)

10 FIG. 1 402 404 10 302 202 302 204 402 404 402 404 402 is a flow chart showing a series of steps of a process performed in the live-streaming systemwhen an active user gives an out-of-stream gift to a streamer. The active user inputs at a user terminal a view request for another user's profile or for an archive of another user's past live-streams, and the out-of-stream UI control unitof the user terminal receives such a request. The out-of-stream communication unitgenerates an information provision request corresponding to the received request, and transmits the information provision request to the serverover the network NW. The streaming information providing unitreceives the information provision request transmitted thereto (S). The streaming information providing unitretrieves the requested information and transmits the retrieved information to the user terminal of the requesting active user (S). Based on the received information, the out-of-stream UI control unitof the user terminal generates a screen corresponding to the request made by the active user and shows the generated screen on the display of the user terminal. For example, in the case of a view request for a profile, the information provision request includes the user ID of the user whose profile is to be retrieved. The information received by the out-of-stream communication unitincludes the profile of the user identified by the user ID included in the information provision request. Based on the received profile, the out-of-stream UI control unitgenerates a profile screen and shows the generated screen on the display. In the case of a view request for an archived live-stream, the information provision request includes identifying information identifying the archived live-stream designated by the active user. The information received by the out-of-stream communication unitincludes the data of the archived live-stream identified by the identifying information included in the information provision request. The out-of-stream UI control unitplays back the archived live-stream based on the received data.

402 404 10 308 206 The active user issues at the user terminal an instruction to use an out-of-stream gift on the profile screen or while the archived live-stream is played back. The out-of-stream UI control unitreceives the instruction. The out-of-stream communication unitgenerates an out-of-stream gift use signal indicating that the active user is to use an out-of-stream gift to the streamer and transmits the generated out-of-stream gift use signal to the serverover the network NW. The out-of-stream gift processing unitreceives the out-of-stream gift use signal (S). The out-of-stream gift use signal includes the user ID of the active user who has used the out-of-stream gift, the user ID of the streamer who receives the out-of-stream gift, and a gift ID identifying the out-of-stream gift.

310 208 310 320 310 318 In response to reception of the out-of-stream gift use signal, the payment processing unitprocesses payment for the price points of the out-of-stream gift by the active user who has given the out-of-stream gift (S). Specifically, the payment processing unitrefers to the gift DBto identify the price points of the out-of-stream gift identified by the gift ID included in the out-of-stream gift use signal. The payment processing unitthen updates the user DBto subtract the identified price points from the points associated with the user ID of the active user included in the out-of-stream gift use signal.

308 318 210 308 320 308 318 The out-of-stream gift processing unitupdates the user DBto increase the reward of the streamer who receives the out-of-stream gift, depending on the reward to be awarded by the out-of-stream gift identified by the gift ID included in the out-of-stream gift use signal (S). Specifically, the out-of-stream gift processing unitrefers to the gift DBto identify the reward to be awarded corresponding to the gift ID included in the received out-of-stream gift use signal. The out-of-stream gift processing unitthen updates the user DBto add the identified reward to the reward associated with the user ID of the streamer included in the out-of-stream gift use signal.

308 330 212 308 330 308 330 10 The out-of-stream gift processing unitupdates the out-of-stream gift history DBin response to the reception of the out-of-stream gift use signal (S). Specifically, the out-of-stream gift processing unitregisters, in the out-of-stream gift history DB, information on the out-of-stream gift used by the active user. The out-of-stream gift processing unitadds a record to the out-of-stream gift history DB. The record includes, as the recipient user ID, the giver user ID, the time, the gift ID, and the price points, the user ID of the streamer included in the out-of-stream gift use signal, the user ID of the active user included in the out-of-stream gift use signal, the time when the serverreceives the out-of-stream gift use signal, the gift ID included in the out-of-stream gift use signal, and the price points of the out-of-stream gift identified by the gift ID.

308 330 308 330 The out-of-stream gift processing unitaccesses the out-of-stream gift history DBperiodically, e.g., once a day, and deletes any out-of-stream gifts for which a predetermined period of time has elapsed since they are used. For example, the out-of-stream gift processing unitdeletes a record held in the out-of-stream gift history DBif the difference between the performed time and the current time exceeds 30 days.

(Within Live-Stream)

11 FIG. 1 322 20 214 322 314 is a flow chart showing a series of steps of a process performed in the live-streaming systemwhen a streamer starts live-streaming and a user joins the live-stream. The streaming start receiving unitreceives, from the user terminalof the streamer over the network NW, a live-stream start signal indicating that a live-stream starts (S). The stream start receiving unitregisters in the stream DBa stream ID identifying the live-stream that has started and the streamer ID of the streamer who performs the live-stream.

324 214 216 324 330 214 330 The out-of-stream gift totaling unitobtains information regarding the out-of-stream gifts used for the streamer who has started live-streaming in the step S(S). The out-of-stream gift totaling unitrefers to the out-of-stream gift history DBto retrieve one or more records having as the recipient user ID the streamer ID of the streamer of the live-stream that has started in the step S, and also deletes the retrieved records from the out-of-stream gift history DB.

324 214 218 324 216 324 334 214 324 334 220 5 324 5 334 334 7 FIG. The out-of-stream gift totaling unitcalculates the total of the out-of-stream gifts used for the streamer of the live-stream that has started in the step S(S). The out-of-stream gift totaling unitcalculates, for each giver user ID, the total of the price points included in the records retrieved in the step S. The out-of-stream gift totaling unitcreates a new out-of-stream gift total table, which is associated with the live-stream initiated in the step S. The out-of-stream gift totaling unitenters, in the out-of-stream gift total table, the result of the calculation, that is to say, the giver user IDs and the totals of the price points of the out-of-stream gifts given by the respective users identified by the giver user IDs (S). According to the example in, when a streamer “S” starts live-streaming, the out-of-stream gift totaling unitcalculates the total of the price points of the out-of-stream gifts for the giver user ID “SS” (4500=“4000+500), and registers the calculated total in the out-of-stream gift total table. Hereinafter, the term “out-of-stream gifter” refers to the giver user registered in the out-of-stream gift total tablegenerated in association with the live-stream.

218 308 330 218 The out-of-stream gifts totaled in the step Sare the out-of-stream gifts that are given to the streamer from when the streamer ends the previous live-stream to when the streamer starts the current live stream. Here, it should be noted that the out-of-stream gift processing unitperiodically updates the out-of-stream gift history DB. For this reason, if the previous live-stream takes place more than 30 days ago, the out-of-stream gifts to be totaled in the step Sare those that are given to the streamer within the last 30 days and before the current live stream starts. Since relatively old out-of-stream gifts are excluded from the total, the user and streamer can be saved from getting confused by a joining effect that appears corresponding to an out-of-stream gift the user does not remember.

302 222 302 334 220 The streaming information providing unitidentifies an out-of-stream gifter satisfying a push notification transmission requirement, that is to say, a big out-of-stream gifter. The big out-of-stream gifter is selected from among the out-of-stream gifters since the big out-of-stream gifter has a total of price points of the out-of-stream gifts exceeding a threshold value (S). The threshold value may be set by the administrator to, for example, 100, 1,000, 3,000 and the like. The streaming information providing unitrefers to the out-of-stream gift total tablegenerated and updated in the step Sto identify, as the big out-of-stream gifter, a giver user having a total of price points greater than the threshold value.

302 214 222 224 302 43 5 302 5 43 5 9 FIG. The streaming information providing unitgenerates a push notification including the information regarding the live-stream initiated in the step Sand transmits the generated push notification to the user terminal of the big out-of-stream gifter identified in the step Svia the network NW (S). In the example shown in, the streaming information providing unitidentifies, as the big out-of-stream gifter, giver users “SS” and “SS” having a total of price points greater than the threshold value=3000. The streaming information providing unitgenerates a push notification including a text stating that the streamer “S” has started a live-stream and a link allowing the giver users to join the live-stream, and transmits the generated push notification to the user terminals of the big out-of-stream gifters “SS” and “SS.” In this way, the streamer can selectively transmit a push notification to the big out-of-stream gifters who have given relatively many out-of-stream gifts to the streamer and are thus highly committed to the streamer, while the streamer can use only a limited number of push notifications. Accordingly, the streamer can attract the largest number of viewers with the least number of push notifications. In addition, since the big out-of-stream gifters who have given a relatively large number of out-of-stream gifts are more likely to join live-streams of the streamer, the streamer is less likely to miss thanking the big out-of-stream gifters. From the perspective of the big out-of-stream gifters, they have a better chance of experiencing a special joining effect, which will be described below, because they receive the push notification.

302 214 226 The streaming information providing unitreceives a request made by any of the active users including the out-of-stream gifters to participate in the live-stream initiated in the step S, from the user terminal of the active user over the network NW (S). The participation request is received along the following two possible routes in the present embodiment.

224 (1) Route Originating at the Push Notification Transmitted in the Step S

214 10 In the case of this route, the user terminal that has received the push notification generates a request to participate in the live-stream initiated in the step Sin response to the push notification, and transmits the generated request to the server.

(2) Route Originating at the Live-Stream Selection Screen

302 404 302 314 214 302 402 214 404 10 According to this route, the streaming information providing unitreceives, over the network NW, an information provision request regarding live-streams from the out-of-stream communication unitof the user terminal of the active user. When receiving the information provision request, the streaming information providing unitrefers to the stream DBand generates a list of currently available live-streams. This list includes the live-stream initiated in the step S. The streaming information providing unittransmits the generated list to the requesting user terminal over the network NW. The out-of-stream UI control unitof the requesting user terminal generates a live-stream selection screen based on the received list and shows the live-stream selection screen on the display of the user terminal. If the live-stream initiated in the step Sis selected on the live-stream selection screen, the out-of-stream communication unitgenerates a request to participate in the selected live-stream, and transmits the request to the serverover the network NW.

302 214 302 314 On reception of the request for participation, the streaming information providing unitstarts providing, to the requesting user terminal, the live-stream initiated in the step S. The streaming information providing unitupdates the stream DBto include the user ID of the active user of the requesting user terminal into the viewer IDs of the stream ID of the live-stream. In this way, the active user can successfully join the live-stream.

326 334 326 334 326 326 326 214 226 326 334 228 326 230 230 222 326 230 When the request for participation is accepted, the effect determining unitrefers to the out-of-stream gift total tableto determine whether to display a joining effect to reflect use of an out-of-stream gift by the active user who has issued the participation request. The effect determining unitfirst refers to the out-of-stream gift total tableto determine whether the active user who has requested participation is an out-of-stream gifter. The effect determining unitdetermines that no joining effect is to appear if the active user is not an out-of-stream gifter. In a case where the active user is an out-of-stream gifter, the effect determining unitdetermines to display a joining effect if the total amount of the out-of-stream gifts given by the out-of-stream gifter satisfies a certain criterion. Otherwise, the effect determining unitdetermines not to display a joining effect. When the out-of-stream gifter participates in the live stream initiated in the step S(YES in S), the effect determining unitrefers to the out-of-stream gift total tableto retrieve the total of price points of the out-of-stream gifts given by the out-of-stream gifter who has participated (S). The effect determining unitcompares the retrieved total of price points against a threshold value (S). The threshold values referred to in the steps Sand Smay be set to the same or different values. In the present embodiment, the threshold values are set to the same value. The effect determining unitdetermines not to display a joining effect if the retrieved total of price points is equal to or less than the threshold value (NO in S).

230 326 232 10 326 326 If the retrieved total of price points exceeds the threshold value (YES in S), the effect determining unitdetermines that the out-of-stream gifter is a big out-of-stream gifter and identifies a joining effect associated with the total of price points (S). In the present embodiment, a plurality of different joining effects are provided corresponding to a plurality of different ranges of the total of price points. In one example, the higher the total of price points is, the more impressive the joining effect is. The data used to display the joining effects may be held in the serverand transmitted to the user terminal as needed. Alternatively, the user terminal may retain the data used to display the joining effects. In this case, the effect determining unitmay generate a signal identifying which joining effect is to be displayed and transmit the generated signal to the user terminal. In the present embodiment, it is the user terminal which holds a plurality of pieces of data to display a plurality of joining effects of different levels of impressiveness. The effect determining unitretrieves a joining effect ID of the joining effect associated with the total of price points.

326 232 214 234 232 The effect determining unitgenerates a joining effect identifying signal to allow the joining effect identified in the step Sto appear on the live-stream room screen and transmits the joining effect identifying signal to the user terminals of all users (streamer and viewers) participating in the live-stream initiated in the step S(S). The joining effect identifying signal includes the joining effect ID retrieved in the step S.

For example, the total of price points is related to the joining effect as follows.

Total of Price Points Joining Effect Joining Effect ID 0 to 2999 None 3000-3999 Small Effect of Short G1 Duration 4000- Long Effect of Long Duration G2

5 334 43 326 334 326 1 326 1 43 202 1 43 43 43 9 FIG. Upon a start of a live-stream by a streamer “S,” the out-of-stream gift total tableshown inis created. When the big out-of-stream gifter “SS” joins the live stream, the effect determining unitrefers to the out-of-stream gift total tableto retrieve the total of price points “3500.” The effect determining unitconsults the above-described relation, to retrieve a joining effect ID “G.” The effect determining unitgenerates a joining effect identifying signal including the retrieved joining effect ID “G” and transmits the joining effect identifying signal to the user terminal of the big out-of-stream gifter “SS.” The viewer-side UI control unitof the user terminal causes a small effect of a short duration corresponding to the joining effect ID “G” included in the received joining effect identifying signal to appear on the live-streaming room screen. The joining effect identifying signal is also transmitted to the user terminal of the streamer and to the user terminals of the other viewers, each of which displays the same small effect of a short duration. In this manner, the big out-of-stream gifter “SS” can view the joining effect almost at the same time as the gifter “SS” joins the live-stream. Accordingly, the big out-of-stream gifter “SS” can feel special and recognized by the streamer unlike users who have not given out-of-stream gifts to the streamer. In addition, viewing the joining effect, the streamer can be aware that the big out-of-stream gifter, who has given a predetermined quantity of out-of-stream gifts or more to him/her, has participated in the live-stream. The streamer can thus directly express his/her gratitude to the big out-of-stream gifter within the live-stream.

3 FIG. 304 20 30 302 304 204 30 304 110 100 20 Returning to, the following now describes the other functional units. The relay unitrelays the video data from the streamer-side user terminalto the viewer-side user terminalin the live-stream started by the streaming information providing unit. The relay unitreceives from the viewer-side communication unita signal that represents user input by a viewer during the live-stream or reproduction of the video data. The signal that represents user input may be an object specifying signal for specifying an object displayed on the display of the user terminal, and the object specifying signal includes the viewer ID of the viewer, the streamer ID of the streamer of the live-stream that the viewer watches, and an object ID that identifies the object. When the object is a gift icon, the object ID is the gift ID. The object specifying signal in that case is an in-stream gift use signal indicating that a viewer uses an in-stream gift for a streamer. Similarly, the relay unitreceives from the streamer-side communication unitof the streaming unitin the user terminala signal that represents user input by the streamer during reproduction of the video data, such as an object specifying signal.

306 318 310 The in-stream gift processing unitupdates the user DBto increase the reward of the streamer by an amount determined by the reward to be awarded by the in-stream gift identified by the gift ID included in the in-stream gift use signal. The payment processing unitprocesses payment of the price of the in-stream gift by the viewer in response to reception of the in-stream gift use signal.

328 330 328 332 The ranking unitperiodically refers to the out-of-stream gift history DBto generate ranking information for each registered recipient user. The ranking unitupdates the out-of-stream gift ranking DBbased on the generated ranking information.

12 FIG. 660 10 660 660 662 663 664 is a representative screen image of a profile screendisplayed on the display of the active user's user terminal. The active user operates the user terminal to identify a streamer of interest, and the user terminal communicates with the serverto show a profile screenof the identified streamer on the display. The profile screenhas an iconindicative of the streamer, profile informationsuch as an attribute of the streamer, and thumbnailsrepresenting edited video data generated by recording the past live-streams of the streamer, or an archive of the past live-streams.

10 664 660 10 The streamer generates an archive by recording and editing his or her past live-streams. Once the streamer uploads the generated archive to the server, the thumbnailsrepresenting the uploaded archive are displayed on the streamer's profile screen. Alternatively, the archive may be edited and generated at the server. The technology of generating archives of live-streams is well known and thus not described in detail herein.

13 FIG. 12 FIG. 10 FIG. 666 664 660 402 404 10 664 666 666 668 664 670 672 is a representative screen image of an archive playback screenappearing on the display of the active user's user terminal. When one of the thumbnailsis tapped on the profile screenshown in, the out-of-stream UI control unitand the out-of-stream communication unitreceive from the serverthe archived data corresponding to the tapped thumbnailand plays back the received data on the archive playback screenin the manner described above with reference to. The archive playback screenhas the streamer's past video imageobtained by playing back the archived data represented by the tapped thumbnail, an out-of-stream gift object, and a viewing end button.

14 FIG. 13 FIG. 6 FIG. 674 670 666 10 402 674 676 is a representative screen image of an out-of-stream gifting screenappearing on the display of the active user's user terminal. When the out-of-stream gift objectis tapped on the archive playback screenshown in, the list of available out-of-stream gifts is transmitted from the serverto the user terminal as described above with reference to. Based on the received list, the out-of-stream UI control unitgenerates the out-of-stream gifting screenincluding an out-of-stream gift pop-upincluding gift icons of out-of-stream gifts and shows it on the display of the user terminal.

676 676 678 680 680 676 10 The out-of-stream gift pop-upis an object for receiving an instruction made by the active user to use an out-of-stream gift to the streamer, who owns the archived data being played back. The out-of-stream gift pop-uphas textexplaining the relation between out-of-stream gifts and joining effects for participation in the next live-stream, and gift iconsrepresenting out-of-stream gifts. The out-of-stream gifts represented by the gift iconsdisplayed in the out-of-stream gift pop-upare among the gifts included in the list received by the user terminal from the server.

680 402 680 404 10 680 680 402 By selecting and tapping any of the gift iconsrepresenting the desired out-of-stream gift, the active user can give the out-of-stream gift to the owner of the archived data, i.e., the streamer. The out-of-stream UI control unitidentifies the tapped gift icon. The out-of-stream communication unitgenerates an out-of-stream gift use signal and transmits the generated signal to the serverover the network NW. The out-of-stream gift use signal includes the user ID of the active user who has used the out-of-stream gift, the user ID of the streamer who owns the archived data being played back when the gift iconis tapped, and the gift ID identifying the out-of-stream gift represented by the tapped gift icon. After this, the out-of-stream gift is handled in the above-described manner. As described above, the out-of-stream UI control unitreceives the instruction to use the out-of-stream gift while the active user is viewing the archived live-stream of the streamer.

15 FIG. 14 FIG. 608 674 608 632 202 is a representative screen image of a live-streaming room screenappearing on the display of the user terminal of the big out-of-stream gifter when the big out-of-stream gifter joins the live-stream of the streamer. The active user may give out-of-stream gifts to the streamer via the out-of-stream gifting screenshown inand the total amount of the out-of-stream gifts may exceed the threshold value. Under such circumstances, the active user (i.e., the big out-of-stream gifter) may join or participate in the streamer's live-stream. In this case, the live streaming room screenincluding a joining effectappears on the display. Having participated in the live-stream, the big out-of-stream gifter is now a viewer of the live-stream. For the big out-of-stream gifter who satisfies the criterion regarding the total amount of the out-of-stream gifts, the viewer-side UI control unitcauses the joining effect to be displayed in response to the participation by the big out-of-stream gifter in the streamer's live-stream.

608 610 10 612 616 626 628 630 632 608 610 202 612 616 626 628 630 632 The live-streaming room screenincludes a video imageof the streamer obtained by reproducing the video data received from the server, a gift object, a comment input region, a ranking object, a comment display region, a viewing end button, and a joining effect. The live-streaming room screencan be generated by superimposing, on the video imageobtained by reproducing the video data by the viewer-side UI control unit, other objects such as the gift object, the comment input region, the ranking object, the comment display region, the viewing end buttonand the joining effect.

628 204 10 202 628 628 608 The comment display regionmay include comments entered by the big out-of-stream gifter, comments entered by other viewers, and notifications from the system. The notifications from the system include information on who has given which gift to the streamer, and information on viewers who have joined the live-stream. The viewer-side communication unitreceives, from the server, the comments entered by the other viewers and the notifications issued by the system. The viewer-side UI control unitgenerates the comment display regionincluding the received comments and notifications, and includes the generated comment display regionin the live-streaming room screen.

616 204 616 10 628 616 The comment input regionallows the big out-of-stream gifter to enter a comment. The viewer-side communication unitgenerates a comment input signal that includes the comment entered in the comment input region, and transmits the signal to the serverover the network NW. At the same time, the user terminal updates the comment display regionto display the comment entered in the comment input region.

632 10 632 200 10 204 202 632 632 608 632 608 634 628 634 632 632 The joining effectis an effect identified by the joining effect ID included in the joining effect identifying signal received from the serverin response to the request for participation in the lives-stream transmitted by the user terminal. A still image or animation is rendered to display the joining effect. While the streamer is live-streaming, the viewing unitcauses the display to show the joining effect corresponding to the use of the out-of-stream gift by the big out-of-stream gifter. If the total amount of out-of-stream gifts given to the streamer by the out-of-stream gifter before the streamer starts live-streaming exceeds the threshold value, the joining effect identifying signal is transmitted from the server, and the viewer-side communication unitreceives the joining effect identifying signal. The viewer-side UI control unitretrieves the data of the joining effectidentified by the joining effect ID included in the received joining effect identifying signal, and superimposes the joining effecton the live-streaming room screenso that the joining effectcan appear on the live-streaming room screenat substantially the same time as the big out-of-stream gifter joins or participates in the live-stream. Since the informationon the big out-of-stream gifter is displayed in the comment display regionupon participation of the big out-of-stream gifter, the informationappears at substantially the same time as the joining effectappears. The joining effectmay include the user ID of the big out-of-stream gifter or the giver, or other information identifying the user of interest.

10 330 10 330 10 10 10 632 110 108 632 10 200 As described above, if the active user gives an out-of-stream gift, the serverregisters the giver, the recipient, the gift ID and the time in the out-of-stream gift history DBin association with each other. At the start of the streamer's live-stream, the serverretrieves from the DBthe information on the out-of-stream gifts given to the streamer. The retrieved information includes the information on the out-of-stream gifters. Based on the retrieved information, the servercan thus detect whether any of the out-of-stream gifters has joined the live-stream. Upon detection of the participation, the serverfirst compares the total amount of out-of-stream gifts given by the out-of-stream gifter who has participated with the threshold value. If the former exceeds the latter, the servertransmits the joining effect identifying signal instructing to display a joining effect, to the user terminals participating in the live-stream. In this way, the same joining effect as the joining effectappears on the live-streaming room screen on the user terminal of the streamer who is live-streaming and on the user terminals of all of the viewers participating in the live-stream. In particular, the streamer-side communication unitof the user terminal of the streamer who is live-streaming receives the joining effect identifying signal. While the streamer is live-streaming, the streamer-side UI control unitcauses the display to show the joining effectcorresponding to the use of the out-of-stream gifts given to the streamer. If the total amount of out-of-stream gifts given by the out-of-stream gifter who has participated in the live-stream is equal to or less than the threshold value, the serverdoes not transmit the joining effect identifying signal. In other words, the viewing unitof the user terminal of the out-of-stream gifter causes the display to show no joining effect if the total amount of out-of-stream gifts used by the out-of-stream gifter for the streamer is equal to or less than the threshold value.

16 FIG. 15 FIG. 15 FIG. 16 FIG. 640 202 642 640 608 642 632 1 642 2 2 1 is a representative screen image of a live-streaming room screenshown on the display of the user terminal of the big out-of-stream gifter when the big out-of-stream gifter participates in the live-stream of the streamer. For the big out-of-stream gifter who satisfies the criterion of the total amount of out-of-stream gifts, the viewer-side UI control unitcauses a joining effectto appear in response to the participation by the big out-of-stream gifter in the streamer's live-stream. The live-streaming room screenis configured in the same manner as the live-streaming room screenof, except for the joining effect. The joining effect depends on the quantity of out-of-stream gifts used by the big out-of-stream gifter to the streamer. The joining effectshown in the example in, which is a small effect of short duration, corresponds to the above-mentioned joining effect ID “G.” The joining effectshown in the example in, which is a large effect of long duration, corresponds to the above-mentioned joining effect ID “G.” Specifically, the joining effect corresponding to the joining effect ID “G” is fancier, more impressive, or showier than the joining effect corresponding to the joining effect ID “G.”

17 FIG. 15 16 FIG.or 17 FIG. 17 FIG. 646 646 610 612 626 616 648 630 632 642 616 632 642 648 650 648 650 652 650 652 652 652 650 650 650 is a representative screen image of a live-streaming room screenshown on the display of the user terminal of the big out-of-stream gifter when the big out-of-stream gifter enters a comment. The live-streaming room screenincludes the video imageof the streamer, a gift object, a ranking object, a comment input region, a comment display region, and a viewing end button. The big out-of-stream gifter, who has joined with the joining effectorshown in, enters a text saying “I've come to see!” in the comment input regionafter the joining effectorends. In, the text thus entered appears in the comment display regionas a comment. In the comment display region, the commentis associated with a badgeand a frame, which depends on the quantity of out-of-stream gifts used for the streamer by the big out-of-stream gifter who has entered the comment. The badgeindicates the user ID of the big out-of-stream gifter who has entered the comment. The badgeof the big out-of-stream gifter is differently displayed than the badges of the other viewers. Specifically, the badgeof the big out-of-stream gifter is displayed in a more prominent manner than the badges of the other viewers. For example, the former has a larger size, and/or a different color, and/or a different font than the latter. The frame of the commentof the big out-of-stream gifter is differently displayed than the frames of the comments of the other viewers. Specifically, the frame of the commentof the big out-of-stream gifter is displayed in a more prominent manner, for example, is drawn by thicker lines or has a larger size than the frames of the comments of the other viewers. At the streamer's user terminal and the other users' user terminals, the commentof the big out-of-stream gifter is displayed in the same manner as in.

18 FIG. 15 16 17 FIGS.,, and 654 626 202 204 10 328 10 332 332 328 204 202 654 654 334 is a representative screen image of an out-of-stream gift ranking screenappearing on the display of the viewer's user terminal. If the viewer taps the ranking objecton the screens shown in, the viewer-side UI control unitinterprets the tap as a ranking display request. On reception of the ranking display request, the viewer-side communication unitgenerates a ranking information request including the streamer ID indicative of the streamer and transmits the request to the serverover the network NW. The ranking unitof the serveridentifies the out-of-stream gift ranking DBassociated with the streamer ID included in the ranking information request and retrieves ranking information from the identified out-of-stream gift ranking DB. The ranking unittransmits the retrieved ranking information to the requesting user terminal. The viewer-side communication unitreceives the transmitted ranking information. Based on the received ranking information, the viewer-side UI control unitgenerates the out-of-stream gift ranking screenand shows the generated screen on the display. The out-of-stream gift ranking screenhas rankings determined by the total amount of out-of-stream gifts effective in the live-stream the viewer is currently viewing, and rankings determined by the total amount of the out-of-stream gifts used for the streamer of the live-stream. The former is basically obtained by ranking the out-of-stream gift total tableassociated with the live-stream based on the total of price points. The latter is obtained by ranking the users based on the total of the out-of-stream gifts given to the streamer of the live-stream, without being limited to the currently viewed live-stream.

18 FIG. 654 400 654 654 654 shows an example where the out-of-stream gift ranking screenis shown during the live-stream. Outside the live-stream, however, the out-of-stream processing unitmay also generate the out-of-stream gift ranking screenin the same manner as described above and cause the display to show the generated screen, in response to a request from the active user. In this case, the out-of-stream gift ranking screenallows the active user to know whether s/he can get a joining effect when s/he joins the next live-stream. In addition, the active user is motivated to give many out-of-stream gifts to achieve a high ranking if s/he can enjoy fancy joining affects by achieving a high ranking. This can encourage the active user to use out-of-stream gifts.

1 1 In the above embodiment, the live-streaming systemmay comprise a holding unit for storing various information used for the service provided by the live-streaming system. The holding unit includes a hard disk or semiconductor memory. It is understood by those skilled in the art that each element or component can be realized by a CPU not shown, a module of an installed application program, a module of a system program, or a semiconductor memory that temporarily stores the contents of data read from a hard disk, and the like.

1 According to the live streaming systemrelating to the embodiment, if an active user gives a gift to a streamer outside a live-stream of the streamer, a corresponding joining effect is generated as feedback when the giver or the active user who has given the out-of-stream gift joins the next live-stream of the recipient or the streamer. This can enhance the sense of feedback experienced by the viewer in return for the out-of-stream gift given by the viewer outside a live-stream. The joining effect allows the streamer to know that the viewer who has joined the live-stream is someone who has supported him or her outside a live-stream. In this way, the streamer can directly show his or her gratitude. In addition, the viewer can also feel honored and rewarded. As a result, the viewers can be motivated to give gifts outside a live-stream, and the connection between the streamers and the viewers can be strengthened.

1 In addition, in the live streaming systemrelating to the embodiment, when a viewer participates in a live-stream of a certain streamer, no joining effects are output unless the quantity of out-of-stream gifts that have been used by the viewer for the streamer reaches the threshold value. In this manner, joining effects can keep providing the sense of being special as they avoid appearing too frequently.

1 In the live streaming systemrelating to the embodiment, when the viewer participates in the live-stream, a joining effect is also displayed on the live-streaming room screen of the streamer if the quantity of out-of-stream gifts that have been used by the viewer for the streamer reaches the threshold value. In this way, the streamer is less likely to forget expressing his or her gratitude to the viewer who has given the streamer many out-of-stream gifts. In addition, out-of-stream gifts for a certain streamer are totaled before every live-stream starts. Therefore, the streamer is less likely to express his or her gratitude redundantly, in other words, to forget that the streamer has already expressed his or her gratitude.

1 In the live streaming systemrelating to the embodiment, the larger quantity of out-of-stream gifts a viewer gives to a streamer, the more impressive joining effects the viewer can get. The viewer may thus give many out-of-stream gifts to emphasize his/her engagement to the streamer.

1 In the live streaming systemrelating to the embodiment, the comments posted by viewers who have given more out-of-stream gifts than the threshold value are displayed in a more prominent manner than those posted by viewers who have not. The viewers who have given out-of-stream gifts can thus feel special as their comments are distinguished from the comments made by the viewers who have not given many out-of-stream gifts. In addition, while joining effects are shown only when the corresponding viewers join, the special badges and frames of their comments are displayed every time the viewers make comments. Due to the distinctively shown comments, the viewers can thus enjoy feeling special repeatedly and continuously.

19 FIG. 19 FIG. 900 10 20 30 Referring to, the hardware configuration of an information processing device relating to an embodiment of the disclosure will be now described.is a block diagram showing an example of the hardware configuration of the information processing device according to the embodiment. The illustrated information processing devicemay, for example, realize the serverand the user terminalsandin the embodiment.

900 901 902 903 900 907 909 911 913 915 917 919 921 925 929 900 901 900 The information processing deviceincludes a CPU, ROM (Read Only Memory), and RAM (Random Access Memory). The information processing devicemay also include a host bus, a bridge, an external bus, an interface, an input device, an output device, a storage device, a drive, a connection port, and a communication device. In addition, the information processing deviceincludes an image capturing device such as a camera (not shown). In addition to or instead of the CPU, the information processing devicemay also include a processing circuit such as a DSP (Digital Signal Processor) or ASIC (Application Specific Integrated Circuit).

901 900 902 903 919 923 901 10 20 30 902 901 903 901 901 902 903 907 907 911 909 The CPUfunctions as an arithmetic processing device and a control device, and controls all or some of the operations in the information processing deviceaccording to various programs stored in the ROM, the RAM, the storage device, or a removable recording medium. For example, the CPUcontrols the overall operation of each functional unit included in the serverand the user terminalsandin the embodiment. The ROMstores programs including sets of instructions, calculation parameters, and the like used by the CPU. The RAMserves as a primary storage that stores programs including sets of instructions to be used in the execution of the CPU, parameters that appropriately change in the execution, and the like. The CPU, ROM, and RAMare interconnected to each other by the host buswhich may be an internal bus such as a CPU bus. Further, the host busis connected to the external bussuch as a PCI (Peripheral Component Interconnect/Interface) bus via the bridge.

915 915 927 900 915 901 915 900 The input devicemay be a user-operated device such as a mouse, keyboard, touch panel, buttons, switches and levers, or a device that converts a physical quantity into an electric signal such as a sound sensor typified by a microphone, an acceleration sensor, a tilt sensor, an infrared sensor, a depth sensor, a temperature sensor, a humidity sensor, and the like. The input devicemay be, for example, a remote control device utilizing infrared rays or other radio waves, or an external connection devicesuch as a mobile phone compatible with the operation of the information processing device. The input deviceincludes an input control circuit that generates an input signal based on the information inputted by the user or the detected physical quantity and outputs the input signal to the CPU. By operating the input device, the user inputs various data and instructs operations to the information processing device.

917 917 917 900 The output deviceis a device capable of visually or audibly informing the user of the obtained information. The output devicemay be, for example, a display such as an LCD, PDP, or OELD, etc., a sound output device such as a speaker and headphones, and a printer. The output deviceoutputs the results of processing by the information processing deviceas text, video such as images, or sound such as audio.

919 900 919 919 901 The storage deviceis a device for storing data configured as an example of a storage unit of the information processing device. The storage deviceis, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or an optical magnetic storage device. This storage devicestores programs executed by the CPU, various data, and various data obtained from external sources.

921 923 900 921 923 903 921 923 The driveis a reader/writer for the removable recording mediumsuch as a magnetic disk, an optical disk, a photomagnetic disk, or a semiconductor memory, and is built in or externally attached to the information processing device. The drivereads information recorded in the mounted removable recording mediumand outputs it to the RAM. Further, the drivewrites record in the attached removable recording medium.

925 900 925 925 927 925 900 927 The connection portis a port for directly connecting a device to the information processing device. The connection portmay be, for example, a USB (Universal Serial Bus) port, an IEEE1394 port, an SCSI (Small Computer System Interface) port, or the like. Further, the connection portmay be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like. By connecting the external connection deviceto the connection port, various data can be exchanged between the information processing deviceand the external connection device.

929 929 929 929 929 929 The communication deviceis, for example, a communication interface formed of a communication device for connecting to the network NW. The communication devicemay be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (trademark), or WUSB (Wireless USB). Further, the communication devicemay be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communications, or the like. The communication devicetransmits and receives signals and the like over the Internet or to and from other communication devices using a predetermined protocol such as TCP/IP. The communication network NW connected to the communication deviceis a network connected by wire or wirelessly, and is, for example, the Internet, home LAN, infrared communication, radio wave communication, satellite communication, or the like. The communication devicerealizes a function as a communication unit.

The image capturing device (not shown) is, for example, a camera for capturing an image of the real space to generate the captured image. The image capturing device uses an imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) and various elements such as lenses that are provided to control image formation of a subject on the imaging element. The image capturing device may capture a still image or may capture a moving image.

1 The configuration and operation of the live-streaming systemin the embodiment have been described. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible by combining the respective components and processes, and that such modifications are also within the scope of the present disclosure.

According to the embodiment, the joining effect is displayed upon participation of the active user in the live-stream of the streamer, but the present inventions is not limited to such. For example, an effect corresponding to the use of the out-of-stream gift may appear at any timing, such as immediately after the start of the live-stream or after a predetermined period of time has elapsed from the start. In this case, the effect may be displayed even when the giver of the out-of-stream gift is not participating in the live-stream.

20 FIG. 20 FIG. 682 682 682 684 5 686 688 689 684 686 688 682 5 is a representative screen image of a live-streaming room screenshown on the display of the viewer's user terminal in a modification example. According to the modification example, at a start of a live-stream by a streamer, the live-streaming room screenappears, which shows the information regarding the out-of-stream gifts given to the streamer after the last live-stream and before the current live-stream. According to the example shown in, the live-streaming room screenhas an objectrepresenting an out-of-stream gift given to the streamer by a viewer “SS”, an objectrepresenting an out-of-stream gift given to the streamer by a viewer “AAA”, an objectrepresenting an out-of-stream gift given to the streamer by a viewer “BBB”, and the number of userswho have given the out-of-stream gifts to the streamer. In this case, the objects,andare displayed in the live-streaming room screeneven if the viewers “SS”, “AAA” and “BBB” are not participating in the live-stream.

204 10 334 202 684 686 688 689 682 684 686 688 The viewer-side communication unitof the viewer's user terminal retrieves, from the serverover the network NW, the information held in the out-of-stream gift total tableassociated with the live-stream in which the viewer is participating. Based on the retrieved information, the viewer-side UI control unitdisplays the objects,,and the number of userson the live-streaming room screen. The objects,andrespectively correspond to the first, second and third places in the rankings of the users determined by the total of price points of the out-of-stream gifts.

689 682 202 626 654 20 FIG. If the viewer taps the number of userson the live-streaming room screenin, the viewer-side UI control unitperforms the same series of steps as when the viewer taps the ranking object, which is described above, to cause the display to show the out-of-stream gift ranking screen.

20 FIG. The objects appearing as shown inallow the users participating in the live-stream to know who support the streamer off-line (outside the live-stream). If any of the viewers who have given the out-of-stream gifts joins the live-stream at a later timing, the streamer can become aware of the participation through the displayed objects and directly express his/her gratitude to the viewer. Furthermore, if a large number of such objects are displayed, this may indicate that the streamer is supported by a large number of users outside the live-stream.

684 686 688 684 686 688 In the above-described modification example, the objects,andmay appear at the start of the live-stream and remain displayed, or they may appear in the middle of the live-stream. Alternatively, the viewers or streamer may be allowed to determine when the objects,andappear.

The live-streaming system may be configured such that, when two streamers are live streaming simultaneously (or in parallel), viewers of one of the live-streams can send out-of-stream gifts to the streamer of the other live-stream.

In the embodiment, the users give out-of-stream gifts while viewing archived data, but the present invention is not limited to such. For example, the profile screen may be configured such that the out-of-stream gifts can be given via the profile screen. Alternatively or additionally, the out-of-stream gifts may be given via the timeline of the streamer, a chat with the streamer, a direct message to the streamer. Out-of-stream gifts refer to any gifts given by users to the streamer while the users have the live-streaming app activated but are not participating in the live-stream of the streamer. For example, while the streamer is live-streaming, the users can give out-of-stream gifts to the streamer by intentionally choosing not to participate in the live-stream.

According to the embodiment, if the total amount (or accumulated amount) of out-of-stream gifts given by a viewer to a certain streamer exceeds a threshold value, a joining effect (for example, animation indicating that the viewer joins a live-stream) appears when the viewer joins the next live-stream of the streamer. If the total amount does not exceed the threshold value, no joining effect appears. The present invention, however, is not limited to such. For example, no threshold value may be set for the total amount of out-of-stream gifts, and any viewers may be entitled to joining effects as long as they have given out-of-stream gifts. Alternatively, the same joining effect may be uniformly offered to any viewers who have given out-of-stream gifts (the same joining effect regardless of the total amount of out-of-stream gifts), and no joining effect may appear for viewers who have not given out-of-stream gifts. Alternatively, there may be two types of out-of-stream gifts: a first type that provides for joining effects; and a second type that does not. In this case, if the active user gives out-of-stream gifts of the first type to a certain streamer, joining effects appear when the active user joins the next live-stream of the streamer. If the active user gives out-of-stream gifts of the second type to a certain streamer, no joining effects appear when the active user joins the next live-stream of the streamer. This can provide for a suitable out-of-steam gift mechanism for users who want to support streamers discreetly. Alternatively, joining effects may appear depending on how many times out-of-stream gifts are given.

According to the embodiment, the out-of-stream gifts are totaled at a start of a live-stream. The present invention, however, is not limited to such. For example, when an active user joins a live-stream, out-of-streams gifts given by the active user up until the time of joining may be totaled.

According to the embodiment, the rankings are based on the total amount of out-of-stream gifts, but the present invention is not limited to such. For example, the rankings may be based on the number of times the out-of-stream gifts are given.

According to the embodiment, different joining effects appear depending on the total amount of out-of-stream gifts, but the present invention is not limited to such. For example, different joining effects may correspond to the respective types of out-of-stream gifts. Alternatively, rankings may be created based on the total amount of out-of-stream gifts, and different joining effects may be provided for the respective places.

650 650 650 According to the embodiment, the commentis displayed with a badge and/or in a frame determined by the quantity of out-of-stream gifts used by the viewer who has entered the commentfor the streamer, but the present invention is not limited to such. For example, the commentmay be displayed with a badge and/or in a frame determined by the type of out-of-stream gifts.

10 According to the embodiment, once the out-of-stream gifts are given to the streamer, a corresponding reward is immediately awarded to the streamer, but the present invention is not limited to such. For example, the servermay be configured such that giving an out-of-stream gift does not automatically result in awarding a reward. Alternatively, the reward corresponding to the out-of-stream gift may be awarded to the streamer provided that the recipient or streamer starts live-streaming, or that the recipient or streamer starts live-streaming and the giver or active user joins the live-stream. In this case, the out-of-stream gifts may offer a higher return rate (the ratio of the reward to be awarded to the price points) for the streamer than in-stream gifts.

According to the embodiment, the joining effects are visual, but the present invention is not limited to such. For example, the joining effects may include audio or vibration, or any combination of visual, audio and vibrational effects.

1 The return rate of the gift, which indicates the ratio of the reward to be awarded to the price points in the embodiment is merely an example, and the return rate may be appropriately set by the administrator of the live-streaming system, for example.

The technical idea according to the embodiment may be applied to live commerce or virtual live-streaming using an avatar that moves in synchronization with the movement of the streamer instead of the image of the streamer.

The procedures described herein, particularly those described with a flow diagram, a flowchart, are susceptible of omission of part of the steps constituting the procedure, adding steps not explicitly included in the steps constituting the procedure, and/or reordering the steps. The procedure subjected to such omission, addition, or reordering is also included in the scope of the present disclosure unless diverged from the purport of the present invention.

10 10 20 30 20 30 20 30 10 10 At least some of the functions realized by the servermay be realized by a device(s) other than the server, for example, the user terminalsand. At least some of the functions realized by the user terminalsandmay be realized by a device(s) other than the user terminalsand, for example, the server. For example, the superimposition of a predetermined frame image on an image of the video data performed by the viewer's user terminal may be performed by the serveror may be performed by the streamer's user terminal.