Cooperative photographing method and apparatus, electronic device, and computer-readable storage medium

This application is the national phase of International Patent Application No. PCT/CN2021/116519, titled “COOPERATIVE PHOTOGRAPHING METHOD AND APPARATUS, ELECTRONIC DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM”, filed on Sep. 3, 2021, which claims priority to Chinese Patent Application No. 202010924413.4, titled “COOPERATIVE PHOTOGRAPHING METHOD AND APPARATUS, ELECTRONIC DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM”, filed on Sep. 4, 2020, both of which are incorporated herein by reference in their entireties.

The present disclosure relates to the technical field of image processing, and in particular to a cooperative photographing method, a cooperative photographing apparatus, an electronic device and a computer-readable storage medium.

With the development of mobile Internet and the popularity of mobile terminals, more and more users spontaneously create and upload some contents to a social platform to share with others. Usually, a content producer uses a photographing device of a mobile terminal to capture and upload interesting images and videos to the social platform to share with other users. However, the current cooperative photographing method lacks diversity and interactivity.

This summary is provided to introduce the idea in a simplified form. The idea will be described in detail in the following description. This summary is neither intended to identify key features or essential features of the claimed technical solution, nor intended to be used to limit the scope of the claimed technical solution.

In a first aspect, the embodiments of the present disclosure provide a cooperative photographing method, which includes: acquiring a first image captured by a first terminal and a second image captured by a second terminal; and composing a third image based on the first image and the second image, where in the third image, a first object in the first image and a second object in the second image are used as foreground objects and superimposed on a specified background image, the first object is a foreground object of the first image, and the second object is a foreground object of the second image.

In a second aspect, the embodiments of the present disclosure provide a cooperative photographing apparatus, which includes: an image acquisition module, configured to acquire a first image captured by a first terminal and a second image captured by a second terminal; and an image composition module, configured to compose a third image based on the first image and the second image, where in the third image, a first object in the first image and a second object in the second image are used as foreground objects and superimposed on a specified background image, the first object is a foreground object of the first image, and the second object is a foreground object of the second image.

In a third aspect, the embodiments of the present disclosure provide an electronic device, which includes one or more computer programs, where the one or more computer programs are stored in a memory and configured to be executed by one or more processors, and the one or more computer programs are configured to perform the method described in the first aspect above.

In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium storing a computer program. When the computer program is invoked and executed by a processor, the method described in the first aspect above is implemented.

With the cooperative photographing method, apparatus, electronic device and computer-readable storage medium provided by the embodiments of the present disclosure, the first image captured by the first terminal and the second image captured by the second terminal are acquired, and then the third image is composed based on the first image and the second image, so that in the third image, the first object in the first image and the second object in the second image are used as foreground objects and superimposed on the specified background image, the first object is the foreground object of the first image, and the second object is the foreground object of the second image. Thus, with the embodiments of the present disclosure, images captured by the first terminal and the second terminal are acquired respectively, and through composition, the first object in the first image and the second object in the second image are used as foreground objects and superimposed on the specified background image to obtain the third image, so that cooperative photographing can be realized based on multiple terminals. Even if users are in different geographical locations, they can still photograph a group photo by their terminals, thereby breaking the space limitation, enriching the social interaction of photographing, and improving the user experience. In addition, the embodiments of the present disclosure also reduce the cost of video production, and can combine images from multiple terminals in real time while photographing, without requiring post production, thus improving the creation efficiency.

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. The embodiments are provided for a more thorough and complete understanding. It should be understood that the drawings and embodiments of the present disclosure are only provided as examples, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit an illustrated step. The scope of the present disclosure is not limited in this regard.

As used herein, the terms “including” and “comprising” and variations thereof are non-exclusive, i.e., meaning “including but not limited to”. The term “based on” means “at least partially based on”. The term “an embodiment” means “at least one embodiment”, the term “another embodiment” means “at least one another embodiment”, and the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as “first” and “second” mentioned in the present disclosure are only used to distinguish devices, modules or units, which are not used to limit that the devices, modules or units are different devices, modules or units and are not used to limit the order or interdependence of functions performed by these devices, modules or units.

It should be noted that the modifications of “a” and “a plurality” mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless clearly indicated otherwise, they should be understood as “one or more”.

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of these messages or information.

The technical solution of the present disclosure and how the technical solution of the present disclosure solves the above technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present disclosure will be described below with reference to the accompanying drawings.

shows a schematic diagram of an implementation environment for applying embodiments of the present disclosure. The implementation environment includes a first terminaland a second terminal.

The first terminaland the second terminalmay be mobile phones, tablet computers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, wearable devices, on-board devices, augmented reality (AR)/virtual reality (VR) devices, laptops, ultra mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) or special cameras (such as a SLR camera, a card camera), which are not limited in the embodiments of the present disclosure.

In addition, the first terminaland the second terminalmay be two terminals of the same type or two terminals of different types, which is not limited in the embodiments of the present disclosure.

The first terminalruns a first client and the second terminalruns a second client. The first client and the second client may be client applications corresponding to a cooperative photographing platform, or other client applications having photographing functions. For example, the first client and the second client may be applications corresponding to short video platforms, social platforms or other platform that support image and cooperative photographing functions.

In some embodiments, the first terminaland the second terminalmay be directly connected through a wired network or wireless network without using a server. In this case, the first terminaland the second terminalmay send captured images to each other, locally generates and display a composite image.

In other embodiments, the first terminaland the second terminalmay be in communication connection through a server. In this case, the implementation environment for applying the embodiments of the present disclosure further includes a server, which can be a traditional server, a cloud server, and may be a single server, a server cluster consisting of several servers, or a cloud computing service center. The first terminalmay be connected with the second terminalthrough the server. The servermay be connected with the first terminaland the second terminalthrough a wired network or a wireless network to realize data interaction.

The embodiments of the present disclosure can be applied to the above terminals or server. If the execution subject is a server, the servermay compose an image and send the image to the first terminaland the second terminalfor display. If the executing body is a terminal, such as the first terminaland/or the second terminal, the terminal may compose an image based on an image captured by the terminal and an image captured by the other terminal. In an embodiment, the image can be composed by only one of the terminals and sent to the other terminal. For example, the first terminalcomposes the image and sends the composed image to the second terminal. In another embodiment, each terminal may perform composition locally.

In some embodiments, the first terminaland/or the second terminaldisplays the composed image.

The following describes in detail the cooperative photographing method, the cooperative photographing apparatus, the electronic device and the computer-readable storage medium provided by the embodiments of the present disclosure.

is a flowchart of a cooperative photographing method according to an embodiment of the present disclosure, which may be applied to the above terminal or server. Taking the execution subject being the first terminal as an example, the process shown inis described in detail. The cooperative photographing method may include the following steps Sand S.

S: Acquire a first image captured by a first terminal and a second image captured by a second terminal.

The first image may be an original image captured by the first terminal through an image acquisition device, or an image obtained by processing the original image. The image acquisition device can be a camera integrated in the first terminal, such as a front camera or a rear camera. Alternatively, the image acquisition device may be an external device connected to the first terminal in a wireless or wired manner, such as an external camera, which is not limited here. The second image is similar to the first image, and one may refer to the description of the first image, which will not be repeated here.

In some embodiments, before step S, the first terminal may generate a cooperative photographing request in responses to a received cooperative photographing trigger instruction inputted by a user, and send the cooperative photographing request to the second terminal, so that the second terminal sends the second image to the first terminal in response to the cooperative photographing request. The cooperative photographing request is used to initiate a cooperative photographing invitation, inviting the other user to shoot an image or a video in cooperation with the user of the first terminal. If the other user agrees with the cooperative photographing invitation corresponding to the cooperative photographing request, the other user confirms the cooperative photographing request through the second terminal and sends a captured image to the first terminal in response to the cooperative photographing request.

For ease of understanding, the first terminal mentioned in the present disclosure represents the initiator of the cooperative photographing request, the second terminal represents the receiver of the cooperative photographing request, the user of the first terminal is referred to as the first user, and the user of the second terminal is referred to as the second user.

If the first terminal detects the cooperative photographing trigger instruction input by the first user, it generates a cooperative photographing request according to the cooperative photographing trigger instruction, and sends the cooperative photographing request to at least one second terminal. The second terminal may confirm the cooperative photographing request, and send the second image captured by the second terminal to the first terminal, so that the first terminal obtain the first image captured by the first terminal and the second image captured by the second terminal.

In process that the first terminal sends the cooperative photographing request to at least one second terminal, the first terminal may first send the cooperative photographing request to the server, and then the server forwards the cooperative photographing request to the second terminal, or the first terminal directly send the cooperative photographing request to the second terminal, which is not limited in embodiments of the present disclosure.

The second terminal can be one or more terminals, that is, the first terminal may send the cooperative photographing request to one or more second terminals, and can perform cooperative photographing with the one or more second terminals at the same time. In addition, the second user may or may not be a contact person of the first user. The second user may be a user who has a friend relationship with the first user in a same application or platform, or may be a unilaterally following user of the first user, or a user unilaterally followed by the first user, or may be any user who does not have a following or friend relationship with the first user, which is not limited in the embodiments of the present disclosure.

In some embodiments, a triggering control element for the cooperative photographing request may be displayed on a screen of the first terminal. If a trigger on the trigger control element, i.e., the cooperative photographing trigger instruction, is detected, the cooperative photographing request is generated. The first terminal may display a sending page for the cooperative photographing request, in which multiple pieces of receiver information to be selected are displayed. The receiver information includes user information and/or platform information. The user information includes at least one of a user avatar and a user name, and the platform information includes at least one of a platform icon and a platform name, which is not limited herein.

In some exemplary embodiments, if the first terminal detects a selection operation on user information, the first terminal may sends the cooperative photographing request to a user terminal corresponding to the selected user information. That is, the first terminal may invite the selected terminal to perform the cooperative photographing, so as to invite the user corresponding to the selected terminal to perform the cooperative photographing. If the first terminal detects a selection operation on platform information, the first terminal may send the cooperative photographing request to a server of a platform corresponding to the platform information, so that the server of the platform sends the cooperative photographing request sent by the first terminal to a number of unspecified second terminals, thereby realizing arbitrary cooperative photographing.

S: Compose a third image based on the first image and the second image, where in the third image, a first object in the first image and a second object in the second image are used as foreground objects and superimposed on a specified background image, the first object is a foreground object of the first image, and the second object is a foreground object of the second image.

The specified background image may be a default image, or any image selected by the user, or a background image of the first image or a background image of the second image, which is not limited in the embodiments. Therefore, more photographing options are provided, so that users can easily replace the background, improving the creation efficiency and interestingness.

In some embodiments, the background image may be specified by the second terminal. As an embodiment, the second terminal may determine the specified background image according to an input of the second user, and send a corresponding background designation instruction to the first terminal. The first terminal may determine the image indicated by the background designation instruction as the specified background image.

The background designation instruction may carry an image identifier according to the input of the second user, and the image corresponding to the image identifier is used as the specified background image. Alternatively, the background designation instruction may carry a terminal identifier, so the background image of the image captured by the terminal corresponding to the terminal identifier is used as the specified background image. For example, if the terminal identifier is “device1”, the background image of the first image captured by the first terminal corresponding to the terminal identifier “device1” is used as the specified background image.

In some other embodiments, the background image may be designated by the first terminal. As an embodiment, the first terminal may determine a background image for the current cooperative photographing according to the cooperative photographing trigger instruction. For example, the first terminal may display one or more cooperative photographing control elements for triggering the cooperative photographing trigger instruction. In this case, different control elements correspond to different background images. For example, control element 1 may take a default image as the specified background image, control element 2 may take a background image of the terminal triggering the cooperative photographing request as the specified background image by default, and control element 3 may specify the background image according to a user selection. Then, according to the cooperative photographing control element triggered by the user, the first terminal can obtain the corresponding cooperative photographing trigger instruction, and determine the background image for this cooperative photographing according to the cooperative photographing trigger instruction.

In some embodiments, the specified background image may be a fourth image selected by a user, where the user may be the first user or the second user. That is, the background image may be selected by any user. In addition, the terminal may preset multiple images, such as various images taken at a rooftop, a downtown, an alley or other places, so that the user can select one or more images as the fourth image according to as needed, and use the fourth image as the specified background image for the composition. Alternatively, multiple images may be acquired through the network for users to choose, which is not limited herein.

In some embodiments, the first terminal may perform image recognition processing on the first image to obtain the foreground object of the first image, i.e., the first object, and perform image recognition processing on the second image to obtain the foreground object of the second image, i.e., the second object. The first object and the second object are superimposed on the specified background image as foreground objects, so that the third image contains both the images captured by the first terminal and the second terminal. In this way, multiple terminals can perform cooperative photographing without limitations of space. Since the background images of the cooperative photographing can be selected by users, the first terminal and the second terminal can perform cooperative photographing based on various background images, which provides more choices for creation, thus meeting the diversity of user needs and realizing convenient background replacement.

In addition, in some embodiments, the first terminal and/or the second terminal may capture an image based on a green screen background, and then perform image recognition processing on the image captured in the green screen background, to extract a foreground object, and superimpose the foreground object on the specified background image to compose the third image, so as to achieve convenient background replacement. In this way, a video can be created with less constraints of reality, so that the user can fully use their creativeness and have more possibilities and freedom in creation, thereby improving the quality of user creations.

In an example scene, some young talented creators have many creative ideas (such as the lightness skill floating on water), but they are unable to create the video due to their limited technical ability. They expect a technology that can break through the restrictions of space and physical laws to achieve various ideas. In this case, they may communicate and perform cooperative photographing in real time based on the green screen background by using the improved cooperative photographing method in this example. Through green screen image cutting and background replacement in real time, the creative cooperative photographing that breaks through the constraints of reality can present at their terminals, which provides more possibilities and freedom in creation.

In other embodiments, the specified background image may also be the background image of the first image or the background image of the second image. In this way, the first object and the second object may be superimposed on the background image of the image captured by one terminal. For example, user A and user B may perform cooperative photographing based on the image captured by the terminal of user B.

In some embodiments, when the specified background image is the background image of the first image or the background image of the second image, the image recognition processing may be performed only on the target image that has not the specified background image to obtain the foreground object of the target image, which is then superimposed on the specified background image. For example, if the specified background image is the background image of the first image, the image recognition processing is performed only on the second image to obtain the second object of the second image, which is then used as a foreground object and superimposed on the first image, so as to compose the third image. For another example, if the specified background image is the background image of the second image, the image recognition processing is performed only on the first image to obtain the first object of the first image, which is then used as a foreground object and superimposed on the first image, so as to compose the third image.

In some other embodiments, when the specified background image is the background image of the first image or the background image of the second image, image recognition processing can still be performed on both the first image and the second image to extract their respective foreground objects to obtain the first object and the second object, which are then used as foreground objects and superimposed on the background image of the first image or the second image.

In some embodiments, while the first terminal and the second terminal capture the first and second images in real time, the first terminal continuously acquires the images captured in real time by the first and second terminals and compose the third images based on the first and second images, so that the images captured in real time by the first and second terminals can be used for the composition in real time. In some embodiments, after the composed third image is obtained, the third image is displayed, and further the real-time composed third image is displayed, so that what users see is what they get during the cooperative photographing process. Thus, when producing a video by the embodiments of the disclosure, the user can watch while photographing, and can obtain the cooperative photographing video without post composition, greatly improving the video production efficiency.

In an example, the first terminal may start a photographing platform application, enable a camera, select at least one second terminal and send a cooperative photographing request to the at least one second terminal. The second terminal confirms to establish a connection with the first terminal after receiving the cooperative photographing request, and also enable a camera. The second terminal transmits the second image captured by the camera of the second terminal to the first terminal. The first terminal also transmits the first image captured by the camera of the first terminal to the second terminal. Then the images captured by the first terminal and the second terminal can be seen on the respective terminals, and finally a video is obtained and recorded. In this way, the real-time cooperative photographing by multiple terminals is achieved, which not only provides a new social interactive method of cooperative photographing, but also allows users to watch while photographing. The cooperative photographing video can be obtained in real time without post composition, thus improving the cooperative photographing efficiency.

With the cooperative photographing method provided in the embodiment, by acquiring the first image captured by the first terminal and the second image captured by the second terminal, and then composing the third image based on the first image and the second image, in the third image the first object in the first image and the second object in the second image are superimposed on the specified background image as foreground objects, where the first object is the foreground object of the first image, and the second object is the foreground object of the second image. In this way, the embodiment of the present disclosure obtains images captured by the first terminal and the second terminal respectively, and composes the third image by superimposing the first object in the first image and the second object in the second image on the specified background image as foreground objects, so that the cooperative photographing can be realized based on multiple terminals. Even if users are in different geographical locations, they can still achieve cooperative photographing in different places by using their own terminals, which breaks through the space limitation, enriches the social interaction method of photographing, and improves the user experience. In addition, the embodiments of the present disclosure also reduce the cost of video production, and can compose images from multiple terminals in real time while photographing without post composition, thereby improving the creation efficiency.