Systems, methods, and computer-readable media are disclosed for dynamically determining audience response to presented content using a video feed. In one embodiment, an example method may include receiving video data for users to which content is presented over a time period, generating, using the video data, a set of frames corresponding to a first user, wherein the set of frames includes a first frame corresponding to a first time during the time period and a second frame corresponding to a second time during the time period, determining, using the first frame and the second frame, a first engagement value for the first user, determining, using the first frame, a first emotional classification for the first user at the first time, determining, using the second frame, a second emotional classification for the first user at the second time, and determining first aggregate user response data for the first user.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method comprising: receiving, by one or more computer processors coupled to at least one memory, a first video feed comprising video data representing an audience to which a visual or auditory performance is presented over a time period, and wherein the first video feed is captured using a video camera in proximity to the audience, and wherein the video data comprises first video frames; partitioning the first video frames into a first plurality of portions and a second plurality of portions, the first plurality of portions comprising a first portion of a first video frame of the first video frames and a first portion of a second video frame of the first video frames, the second plurality of portions comprising a second portion of the first video frame and a second portion of the second video frame, the first portion of the first video frame and the first portion of the second video frame each representing a first separate single member of the audience, the second portion of the first video frame and the second portion of the second video frame each representing a second separate single member of the audience; generating a second video feed comprising the first plurality of portions, the second video feed corresponding to the first separate single member, wherein the first plurality of portions comprises a third video frame comprising the first portion of the first video frame and corresponding to a first timestamp during the time period, and a fourth video frame comprising the first portion of the second video frame and corresponding to a second timestamp during the time period; generating a third video feed comprising the second plurality of portions, the third video feed corresponding to the second separate single member, wherein the second plurality of portions comprises a fifth video frame comprising the second portion of the first video frame and corresponding to the first timestamp, and a sixth video frame comprising the second portion of the second video frame and corresponding to the second timestamp; determining, using the third video frame and the fourth video frame, a first amount of body movement of the first separate single member between the first timestamp and the second timestamp; determining, using the first amount of body movement, a first engagement score for the first separate single member; determining, using the third video frame, a first emotional state classification for the first separate single member at the first timestamp; determining, using the fourth video frame, a second emotional state classification for the first separate single member at the second timestamp; determining first aggregate member response data for the first separate single member, wherein the first aggregate member response data comprises the first engagement score, the first emotional state classification, and the second emotional state classification; determining, using the fifth video frame and the sixth video frame, a second amount of body movement of the second separate single member between the first timestamp and the second timestamp; determining, using the second amount of body movement, a second engagement score for the second separate single member; determining, using the fifth video frame, a third emotional state classification for the second separate single member at the first timestamp; determining, using the sixth video frame, a fourth emotional state classification for the second separate single member at the second timestamp; determining second aggregate member response data for the second separate single member, wherein the second aggregate member response data comprises the second engagement score, the third emotional state classification, and the fourth emotional state classification; and determining aggregate audience response data for the audience, wherein the aggregate audience response data comprises the first aggregate member response data and the second aggregate member response data.
2. The method of claim 1 , wherein the first engagement score is a numerical value or a percentage value, and wherein determining the first amount of body movement of the first separate single member between the first timestamp and the second timestamp comprises: comparing pixels of the third video frame and corresponding pixels of the fourth video frame; and determining an amount of pixel change between the third video frame and the fourth video frame, wherein the amount of pixel change comprises one of: (i) a numerical value corresponding to a number of pixels of the fourth video frame that are different from corresponding pixels of the third video frame; or (ii) a percentage value corresponding to the number of pixels of the fourth video frame that are different from corresponding pixels of the third video frame, divided by a total number of pixels in each of the third video frame and the fourth video frame.
3. The method of claim 1 , wherein the first emotional state classification and the second emotional state classification are textual values, wherein determining the first emotional state classification for the first separate single member at the first timestamp comprises determining, using a pattern recognition algorithm, a first facial expression of the first separate single member at the first timestamp using the third video frame, and wherein determining the second emotional state classification for the first separate single member at the second timestamp comprises determining, using the pattern recognition algorithm, a second facial expression of the first separate single member at the second timestamp using the fourth video frame.
4. The method of claim 1 , further comprising: determining, using the first engagement score and the second engagement score, an average audience engagement score for the audience for a time interval between the first timestamp and the second timestamp; determining that the average audience engagement score is a maximum audience engagement score for the audience during the time period; and causing a presentation of the average audience engagement score and a portion of the visual or auditory performance corresponding to the time interval between the first timestamp and the second timestamp at a user device.
5. A method comprising: receiving, by one or more computer processors coupled to at least one memory, a first video feed comprising video data representing a plurality of users to which content is presented over a time period, the video data comprising first video frames; partitioning the first video frames into a plurality of portions, the plurality of portions comprising a portion of a first video frame of the first video frames and a portion of a second video frame of the first video frames, the portion of the first video frame and the portion of the second video frame representing a separate single user of the plurality of users; generating, a second video feed comprising the plurality of portions, the second video feed corresponding to the separate single user, wherein the plurality of portions comprises a third video frame comprising the portion of the first video frame and corresponding to a first time during the time period, and a fourth video frame comprising the portion of the second video frame and corresponding to a second time during the time period; determining, using the third video frame and the fourth video frame, a first engagement value for the separate single user; determining, using the third video frame, a first emotional classification for the separate single user at the first time; determining, using the fourth video frame, a second emotional classification for the separate single user at the second time; and determining first aggregate user response data for the separate single user, wherein the first aggregate user response data comprises the first engagement value, the first emotional classification, and the second emotional classification.
6. The method of claim 5 , wherein the first engagement value is a numerical value or a percentage value, and wherein determining the first engagement value for the separate single user comprises: determining, using the third video frame and the fourth video frame, a first amount of body movement of the separate single user between the first time and the second time; and determining the first engagement value using the first amount of body movement.
7. The method of claim 6 , wherein determining the first amount of body movement of the separate single user between the first time and the second time comprises: comparing the third video frame and the fourth video frame; and determining an amount of pixel change between the third video frame and the fourth video frame, wherein the amount of pixel change comprises one of: (i) a numerical value corresponding to a number of pixels of the fourth video frame that are different from corresponding pixels of the third video frame; or (ii) a percentage value corresponding to the number of pixels of the fourth video frame that are different from corresponding pixels of the third video frame, divided by a total number of pixels in each of the third video frame and the fourth video frame.
8. The method of claim 5 , wherein determining the first emotional classification for the separate single user at the first time comprises determining a first facial expression of the separate single user at the first time using the third video frame, and wherein determining the second emotional classification for the separate single user at the second time comprises determining a second facial expression of the separate single user at the second time using the fourth video frame.
9. The method of claim 5 , wherein the plurality of portions further comprises a fifth video frame corresponding to a third time during the time period, the method further comprising: determining, using the fourth video frame and the fifth video frame, a second engagement value for the separate single user; and determining, using the fifth video frame, a third emotional classification for the separate single user at the third time.
10. The method of claim 5 , wherein the plurality of users further comprises a second separate single user, and wherein the plurality of portions further comprises a second portion of the first video frame and a second portion of the second video frame, the second portion of the first video frame and the second portion of the second video frame representing the second separate single user, the method further comprising: generating, using the second portion of the first video frame, a third video feed corresponding to the second separate single user, wherein the third video feed comprises a fifth video frame comprising the second portion of the first video frame and corresponding to the first time, and a sixth video frame comprising the second portion of the second video frame and corresponding to the second time; determining, using the fifth video frame and the sixth video frame, a second engagement value for the second separate single user; determining, using the fifth video frame, a third emotional classification for the second separate single user at the first time; determining, using the sixth video frame, a fourth emotional classification for the second separate single user at the second time; and determining second aggregate user response data for the second separate single user, wherein the second aggregate user response data comprises the second engagement value, the third emotional classification, and the fourth emotional classification.
11. The method of claim 10 , further comprising: determining, using the first engagement value and the second engagement value, an average engagement value for the plurality of users; and causing a presentation of the average engagement value and a portion of the content corresponding to the second time at a user device.
12. The method of claim 10 , further comprising: determining, using the second emotional classification and the fourth emotional classification, a top emotional classification for the plurality of users at the second time; and causing a presentation of the top emotional classification and a portion of the content corresponding to the second time at a user device.
13. The method of claim 5 , wherein the time period is a first time period, the method further comprising: receiving a third video feed comprising video data representing one or more additional users to which the content is presented over a second time period, wherein the one or more additional users comprises a second separate single user, and wherein the second time period is after the first time period; generating a fourth video feed corresponding to the second separate single user, wherein the fourth video feed comprises a fifth video frame corresponding to a third time during the second time period and a sixth video frame corresponding to a fourth time during the second time period; determining, using the fifth video frame and the sixth video frame, a second engagement value for the second separate single user; determining, using the fifth video frame, a third emotional classification for the second separate single user at the third time; determining, using the sixth video frame, a fourth emotional classification for the second separate single user at the fourth time; and determining second aggregate user response data for the second separate single user, wherein the second aggregate user response data comprises the second engagement value, the third emotional classification, and the fourth emotional classification.
14. The method of claim 13 , further comprising: syncing the first aggregate user response data and the second aggregate user response data to the content using an elapsed time of the content, wherein the first time and the third time correspond to a first elapsed time of the content, and wherein the second time and the fourth time correspond to a second elapsed time of the content.
15. A device comprising: at least one memory that stores computer-executable instructions; and at least one processor configured to access the at least one memory and execute the computer-executable instructions to: receive, by one or more computer processors coupled to at least one memory, a first video feed comprising video data representing a plurality of users to which content is presented over a time period, the video data comprising first video frames; partition the first video frames into a plurality of portions, the plurality of portions comprising a portion of a first video frame of the first video frames and a portion of a second video frame of the first video frames, the portion of the first video frame and the portion of the second video frame representing a first user of the plurality of users; generate a second video feed comprising the plurality of portions, the second video feed corresponding to the first user, wherein the plurality of portions comprises a third video frame comprising the portion of the first video frame and corresponding to a first time during the time period, and a fourth video frame comprising the portion of the second video frame and corresponding to a second time during the time period; determine, using the third video frame and the fourth video frame, a first engagement value for the first user; determine, using the third video frame, a first emotional classification for the first user at the first time; determine, using the fourth video frame, a second emotional classification for the first user at the second time; and determine first aggregate user response data for the first user, wherein the first aggregate user response data comprises the first engagement value, the first emotional classification, and the second emotional classification.
16. The device of claim 15 , wherein the plurality of users further comprises a second user, wherein the plurality of portions further comprises a second portion of first video frame and a second portion of the second video frame, the second portion of the first video frame and the second portion of the second video frame representing the second user, and wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to: generate, using the second portion of the first video frame, a third video feed corresponding to the second user, wherein the third video feed comprises a fifth video frame comprising the second portion of the first video frame and corresponding to the first time, and a sixth video frame comprising the second portion of the second video frame and corresponding to the second time; determine, using the fifth video frame and the sixth video frame, a second engagement value for the second user; determine, using the fifth video frame, a third emotional classification for the second user at the first time; determine, using the sixth video frame, a fourth emotional classification for the second user at the second time; and determine second aggregate user response data for the second user, wherein the second aggregate user response data comprises the second engagement value, the third emotional classification, and the fourth emotional classification.
17. The device of claim 16 , wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to: determine, using the first engagement value and the second engagement value, an average engagement value for the plurality of users; determine, using the second emotional classification and the fourth emotional classification, a top emotional classification for the plurality of users at the second time; and cause a presentation of the average engagement value, the top emotional classification, and a portion of the content corresponding to the second time at a user device.
18. The device of claim 15 , wherein the time period is a first time period, and wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to: receive a third video feed comprising video data representing one or more additional users to which the content is presented over a second time period, wherein the one or more additional users comprises a second user, and wherein the second time period is after the first time period; generate a fourth video feed corresponding to the second user, wherein the fourth video feed comprises a fifth video frame corresponding to a third time during the second time period and a sixth video frame corresponding to a fourth time during the second time period; determine, using the fifth video frame and the sixth video frame, a second engagement value for the second user; determine, using the fifth video frame, a third emotional classification for the second user at the third time; determine, using the sixth video frame, a fourth emotional classification for the second user at the fourth time; determine second aggregate user response data for the second user, wherein the second aggregate user response data comprises the second engagement value, the third emotional classification, and the fourth emotional classification; and sync the first aggregate user response data and the second aggregate user response data to the content using an elapsed time of the content, wherein the first time and the third time correspond to a first elapsed time of the content, and wherein the second time and the fourth time correspond to a second elapsed time of the content.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
February 21, 2018
December 29, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.