Information Processing Apparatus, Information Processing Method, and Program

The present application is a continuation application of U.S. patent application Ser. No. 17/430,974, filed Aug. 13, 2021, which is a National Stage Entry of PCT/JP2019/006909, filed on Feb. 22, 2019, the entire contents of which are hereby incorporated by reference.

The present disclosure relates to an information processing apparatus, an information processing method, and a program.

In recent years, various kinds of applications with high entertainment value have been developed. Further, technologies which can create various kinds of added values by providing extension functions in cooperation with such applications as described above have also been proposed. For example, PTL 1 discloses a technology for synthesizing an image of a user who is playing a game application, onto a game screen, and for distributing the resulting screen.

As described above, in the case where an extension function is provided to an external application, cooperation is important so as not to cause any interference with the original behavior of such an external application and the user's operation of the application.

According to the present disclosure, there is provided an information processing apparatus that includes a controller configured to control functional extension of an external application. The controller is configured to cause an extended menu icon whose display position is adjustable to be superimposed and displayed on a display window of the external application, and control a display position of an extended menu according to the display position of the extended menu icon; and cause an extension function icon to be superimposed and displayed on the display window with a display position of the extension function icon being adjustable, the extension function icon being associated with an extension function selected on the extended menu, and control a display position of a sub-icon accompanying the extension function icon, according to the display position of the extension function icon.

Further, according to the present disclosure, there is provided an information processing method that includes controlling, by a processor, functional extension of an external application. The controlling includes causing an extended menu icon whose display position is adjustable to be superimposed and displayed on a display window of the external application, and controlling a display position of an extended menu according to the display position of the extended menu icon; and causing an extension function icon to be superimposed and displayed on the display window with a display position of the extension function icon being adjustable, the extension function icon being associated with an extension function selected on the extended menu, and controlling a display position of a sub-icon accompanying the extension function icon, according to the display position of the extension function icon.

Further, according to the present disclosure, there is provided a program that causes a computer to function as an information processing apparatus that includes a controller configured to control functional extension of an external application. The controller is configured to cause an extended menu icon whose display position is adjustable to be superimposed and displayed on a display window of the external application, and control a display position of an extended menu according to the display position of the extended menu icon; and cause an extension function icon to be superimposed and displayed on the display window with a display position of the extension function icon being adjustable, the extension function icon being associated with an extension function selected on the extended menu, and control a display position of a sub-icon accompanying the extension function icon, according to the display position of the extension function icon.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the appended drawings. Note that, in the present specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted.

Note that descriptions will be made in the following order.

First, descriptions will be made regarding an example of the functional configuration of an information processing apparatusaccording to one embodiment of the present disclosure. As described earlier, in recent years, various kinds of applications with high entertainment value have been developed. Examples of such applications as described above include game applications. A user can easily play various game applications by using, for example, a terminal such as a smartphone.

In addition, in recent years, user's enjoyment has not been limited to only playing a game application. Sharing the user's playing status of the game application with others has become popular. For example, the user can transmit, to an acquaintance, a moving image produced by synthesizing an image of a user's face on the recorded screen of playing status of a game application, or can distribute the moving image to an unspecified large number of viewers/listeners via a moving image distribution service or other services. Sharing of such a moving image allows the viewers/listeners to enjoy the playing status of the game application together with the distributer's reaction and the like, and thus enables the experience of gaming to be shared among a plurality of users.

Here, in the case where such a moving image as described above is to be produced, in general, editing work is necessary for synthesizing a user's own image, after the image of the playing status of the game application is recorded and the user's image is photographed simultaneously. However, such editing work requires knowledge regarding moving image editing and the like, in addition to time and labor.

On the other hand, in order to reduce such a burden on the distributor as described above, there is also known a technology for automatically producing a play moving image in which the distributor's own image is synthesized, by cooperating with the game application, as described in PTL 1, for example. By using such a technology, even a distributor having no knowledge regarding moving image editing can easily produce a play moving image of a game application, distribute the resulting image, and do any other related action.

However, in the case where such a technology as described above is used to provide an extension function to an external application, cooperation is important so as not to cause any interference with the original behavior of such an external application and the user's operation of the external application. In addition, it is also desirable that the user is able to operate the external application more intuitively and to flexibly control the extension function according to the situation of the external application.

The technical idea of the present disclosure has been conceived in view of the above-described situation and makes it possible to provide an extension function that permits more intuitive operations without interfering with the behavior of the external application. To this end, the information processing apparatusaccording to the embodiment of the present disclosure includes a controllerthat controls functional extension for an external application. Further, one of the features of the controlleraccording to the embodiment of the present disclosure is that an extended menu icon whose display position is adjustable is displayed and superimposed on the display window of the external application, and that the display position of an extended menu is controlled according to the display position of the displayed-superimposed extended menu icon. In addition, one of the features of the controlleraccording to the embodiment of the present disclosure is that the controllercauses an extension function icon associated with the extension function having been selected on the extended menu described above, to be displayed and superimposed on the display window described above, with the display position of the extension function icon being adjustable, and that the controllercontrols the display position of a sub-icon accompanying the displayed-superimposed extension function icon according to the display position of the extension function icon.

Hereinafter, detailed descriptions will be made regarding an example of the functional configuration of the information processing apparatus, which has the features described above, according to the present embodiment. The information processing apparatusaccording to the present embodiment may be a smartphone, a tablet, a PC (Personal Computer), or the like capable of executing various kinds of applications.

depicts a block diagram illustrating an example of the functional configuration of the information processing apparatusaccording to the present embodiment. As depicted in, the information processing apparatusaccording to the present embodiment includes an operation unit, a sound input unit, an image capturing unit, the controller, a sound output unit, and a display unit.

The operation unitaccording to the present embodiment detects various operations by a user, such as an operation of an application. For this function, the operation unitaccording to the present embodiment includes, for example, a touch panel, a button, a keyboard, a mouse, and the like. The operation unitaccording to the present embodiment inputs, to the controller, information associated with the detected user's operation.

The sound input unitaccording to the present embodiment collects the sound uttered by the user and the like, on the basis of control by the controller. For this function, the sound input unitaccording to the present embodiment includes a microphone and the like.

The image capturing unitaccording to the present embodiment photographs a moving image on the basis of control by the controller. The image capturing unitmay photograph, for example, a face or the like of a user who operates the information processing apparatus. For this function, the image capturing unitaccording to the present embodiment includes an imaging element. The smartphone, which is an example of the information processing apparatus, includes an inward-facing camera that photographs the user's face and the like and is disposed on the display unitside and a main camera that photographs a landscape and the like and is disposed on the back side of the display unit. In the present embodiment, photographing by using the inward-facing camera is controlled, as an example.

The controlleraccording to the present embodiment controls each of the components included in the information processing apparatus. In addition, one of the features of the controlleraccording to the present embodiment is to control the extension of the functions of external applications. Details of the functions of the controlleraccording to the present embodiment will separately be described later.

The sound output unitaccording to the present embodiment outputs various sounds. The sound output unitaccording to the present embodiment outputs sounds according to the situation of an application on the basis of control by the controller, for example. For this function, the sound output unitaccording to the present embodiment includes a loudspeaker and an amplifier.

The display unitaccording to the present embodiment displays various kinds of visual information on the basis of control by the controller. The display unitaccording to the present embodiment may display, for example, images and characters associated with an application. For this function, the display unitaccording to the present embodiment includes various kinds of display devices.

Up to this point, the descriptions have been made regarding the example of the functional configuration of the information processing apparatusaccording to the present embodiment. It should be noted that the configuration described above with reference tois merely an example, and the functional configuration of the information processing apparatusaccording to the present embodiment is not limited to such an example. The functional configuration of the information processing apparatusaccording to the present embodiment may flexibly be modified according to specifications and operation.

Next, detailed descriptions will be made regarding the functions of the information processing apparatusaccording to the present embodiment. One of the features of the controllerof the information processing apparatusaccording to the present embodiment is to control extended applications for providing extension functions to various kinds of external applications. Examples of the external applications described above may include a game application. The controlleraccording to the present embodiment is capable of providing an extension function such as an image-recording function to an external game application produced by a different company, for example, by causing the above-described extended application to cooperate with the external application.

Note that descriptions will hereinafter be made by using a case, as a major example, where the external application associated with the present embodiment is a game application; however, the external application associated with the present embodiment is not limited to such an example. Examples of the external application associated with the present embodiment may include drawing applications, various kinds of editing applications, and applications for viewing/listening moving images, music, etc.

Further, the external application associated with the present embodiment is not limited to an application produced by a different company, and it may be an application produced in-house. By using the extended application according to the present embodiment, it is possible to easily provide an extension function to an already-existing application no matter who its producer is, without editing the source code or the like.

depicts a diagram for illustrating cooperation between the extended application according to the present embodiment and an external application. Both the extended application according to the present embodiment and the external application function by being installed in the information processing apparatus.

On the left side of, there is depicted a home screen of an OS (Operating System) displayed on the display unitof the information processing apparatus. Here, in the case where a user selects an icon Ia associated with an extended application, the controllerstarts the extended application and, as depicted in the center of the figure, causes the display unitto display a start screen of the extended application.

In the case of one example depicted in, on the start screen described above, there are displayed icons Ig and Ih associated with installed external applications that are available for cooperation and icons Ii to Ik associated with not-installed external applications that are recommended to the user. The controlleris capable of automatically producing a list of applications that includes installed external applications and exemplified applications based on recommendation information or other information. Further, such external applications displayed on the start screen may include another one optionally added by the user.

Here, in the case where the user selects an icon associated with an intended external application, as depicted on the right side of the figure, the controllerstarts the external application corresponding to the selected icon, and causes the display unitto display the display window of the intended external application. In, one example of the case is depicted in which the user selects the icon Ig associated with a racing game on the start screen described above.

Further, at this time, the controllercauses an extended menu icon MI to be superimposed and displayed on the display window described above. The extended menu icon MI may be an icon for invoking a menu of extension functions to be provided to the external application. In addition, the user may be allowed to move the extended menu icon MI to any position on the display window by dragging the extended menu icon MI, for example. By appropriately moving the extended menu icon MI according to the situation of display screen of the external application, it is possible to avoid interference with the user's operation of the external application and the display of various kinds of visual information displayed by the external application.

Note that the above description has been made regarding the exemplified case where the user starts the external application to be cooperated with the extended application via the start screen of the extended application. However, for example, the user may be able to directly select the icon associated with the intended external application, on the home screen, so as to start the intended external application in the state of being in cooperation with the extended application.

Next, the display control of the extended menu according to the present embodiment will be described. As described above, the user can invoke an extension function menu by selecting the extended menu icon MI that is superimposed and displayed on the display window of the external application. In such a case, the controlleraccording to the present embodiment controls the display position of the extended menu on the basis of the display position of the extended menu icon MI on the display window.

The controlleraccording to the present embodiment may control the display position of the extended menu on the basis of both the display position of the extended menu icon MI and the aspect ratio of the display window, for example.are diagrams for illustrating the extended menu display based on the display position of the extended menu icon according to the present embodiment.

In general, there are an external application operated with the information processing apparatusheld vertically, an external application operated with the information processing apparatusheld horizontally, and an external application operated while the display of the information processing apparatusis turned according to the posture of the information processing apparatus. For this reason, the controlleraccording to the present embodiment is capable of displaying the extended menu at a position which permits easier operation of the menu, by detecting the aspect ratio of the display window of the external application.

For example, in the case of one example depicted in, the display unitis displaying a display window that is suitable for the operation using the information processing apparatusin the state of being held vertically. In this way, in the case where the display window is in a state where its side along the height direction is longer than its side along the width direction, the controllermay cause an extended menu M to be superimposed and displayed in either an upper region or a lower region of the display window that contains the display position of the extended menu icon MI.

For example, in the case of one example depicted in the upper stage of, the extended menu icon MI is located in the upper region of the display window. Here, in the case where the extended menu icon MI is selected, the controllermay cause the extended menu M to be superimposed and displayed in the upper region of the display window, similarly to the display position of the extended menu icon MI.

On the other hand, in the case of one example depicted in the lower stage of, the extended menu icon MI is located in the lower region of the display window. Here, in the case where the extended menu icon MI is selected, the controllermay cause the extended menu M to be superimposed and displayed in the lower region of the display window, similarly to the display position of the extended menu icon MI.

Further, for example, in the case of one example depicted in, the display unitis displaying a display window that is suitable for operation using the information processing apparatusin the state of being held horizontally. In this way, in the case where the display window is in a state where its side along the width direction is longer than its side along the height direction, the controllermay cause the extended menu M to be superimposed and displayed in either a left-side region or a right-side region of the display window that contains the display position of the extended menu icon MI.

For example, in the case of one example depicted in the upper stage of, the extended menu icon MI is located in the left-side region of the display window. Here, in the case where the extended menu icon MI is selected, the controllermay cause the extended menu M to be superimposed and displayed in the left-side region of the display window, similarly to the display position of the extended menu icon MI.

On the other hand, in the case of one example depicted in the lower stage of, the extended menu icon MI is located in the right-side region of the display window. Here, in the case where the extended menu icon MI is selected, the controllermay cause the extended menu M to be superimposed and displayed in the right-side region of the display window, similarly to the display position of the extended menu icon MI.

Up to this point, the descriptions have been made regarding the display control of the extended menu M based on the display position of the extended menu icon MI according to the present embodiment. According to the display control as described above, the user having performed the operation of the extended menu icon MI can successively select any extension function in the same region, and thus, the extended menu display featuring easier operation can be realized. In addition, according to such a display control as described above, the extended menu M can be displayed in the same region as the display position of the extended menu icon MI that has been moved by the user to the position where the icon does not obstruct user's operations. This can effectively reduce the possibility that important visual information output by the external application is screened from view by the extended menu M.

Note that, in the example depicted in, a case is illustrated in which the functions of setting a game mode, setting focus, searching, taking a screenshot, and recording an image are offered as extension functions.

The function of setting a game mode described above may be an extension function capable of setting, for each external application, a mode such as a mode with priority given to the power saving or a mode with priority given to the performance, for example.

Further, the function of setting focus described above may be an extension function capable of setting various kinds of conditions that can help a user further concentrate on the external application, for example. The user may be able to set functions of releasing a RAM, locking a navigation bar, not displaying notification from other applications such as a message application, and turning-off the function of automatic brightness adjustment of the display unit, for example. The details of the game mode setting and focus setting may be stored after the external application has ended, thereby allowing the setting to be reflected automatically upon starting up the next operation.

Further, the function of searching described above may be an extension function capable of performing various kinds of information retrieval on the display window of the external application, without separately starting the browser application by the user. By use of the searching function, the user can play a game while referring to, on the display window, information for strategy, for example.

Further, the function of taking a screenshot described above may be an extension function capable of storing the playing status of the external application as a still image. In addition, the function of recording an image described above may be an extension function capable of storing the playing status of the external application as a moving image. Details of the function of image-recording according to the present embodiment will separately be described later.

Up to this point, the descriptions have been made regarding the example of the extended menu M and the extension function according to the present embodiment. Note that such an extension function as described above is used by the user according to the situation, so that the extension function is not assumed to be usually frequently operated. For this reason, the extended menu icon MI for invoking the extended menu M as described above is preferably displayed so as not to interfere with the playing of the external application as much as possible in the case where the user's operation is not given.