Systems and Methods for Mitigating the Spread of Offensive Content And/Or Behavior

1. A method for identifying offensive message content comprising: for each particular responsive message of a plurality of responsive messages received in response to an initial message: providing, by one or more computers, content of the particular responsive message as an input to a machine learning model that has been trained to predict a likelihood that an initial message provided by a first user device includes offensive content based on processing of responsive message content from a different user device provided in response to the initial message; processing, by one or more computers, the content of the particular responsive message through the machine learning model to generate output data indicating a likelihood that the initial message includes offensive content; and storing, by one or more computers, the generated output data; determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content; and based on a determination, by one or more computers, that the output data generated for each of the plurality of responsive messages indicates that the initial message likely includes offensive content, performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content, wherein performing the one or more remedial operations comprises: adjusting, using one or more computers, a content score associated with the initial message content, wherein the adjusted content scores causes the initial message content to be demoted in list of content items.

2. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: deleting, using one or more computers, the initial message, wherein deletion of the initial message content prohibits any other user from viewing the initial message content after its deletion.

3. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: flagging, using one or more computers, the initial message for deletion.

4. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: storing, using one or more computers, the content of the initial message in a database of offensive content used to screen messages or other content for offensive content.

5. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: training, using one or more computers, a machine learning model to detect subsequent messages that have the initial message content as offensive content.

6. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: causing, using one or more computers, a warning message to be displayed in proximity to the initial message content within a messaging application.

7. The method of claim 1, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content comprises: for each particular instance of output data generated for a responsive message: determining, by one or more computers, whether the particular instance of output data satisfies a predetermined threshold; and incrementing, using one or more computers, one of a plurality of counters based on the determination as to whether the particular instance of output data satisfies a predetermined threshold, wherein incrementing one of the plurality of counters based on the determination comprises: incrementing a first counter corresponding to a first determination that the particular responsive message indicates that the initial message is likely offensive, or incrementing a second counter corresponding to a second determination that the particular responsive message indicates that the initial message is not likely offensive.

8. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely includes offensive content if the first counter is greater than the second counter.

9. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely includes offensive content if the first counter is greater than the second counter after evaluation of a threshold number of responsive messages.

10. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely includes offensive content if the first counter satisfies a predetermined threshold number of occurrences.

11. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely does not include offensive content if the second counter is greater than the first counter.

12. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely does not include offensive content if the second counter is greater than the first counter after evaluation of a threshold number of responsive messages.

13. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the initial message likely does not include offensive content if the second counter satisfies a predetermined threshold number of occurrences.

14. The method of claim 7, wherein determining, by one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content further comprises: determining, by one or more computers, that the first counter and the second counter are equal; and based on a determination, by one or more computers, that the first counter and the second counter are equal, determining, by one or more computers, whether the initial message likely includes offensive content based on one or more of a number of likes associated with the initial message, a number of different types of emojis associated with the initial message, or a number of comments associated with the initial message.

15. The method of claim 1, wherein performing, by one or more computers, one or more remedial operations to mitigate exposure to the offensive content further comprises: generating, using one or more computers, notification data that, when processed by the first user device, causes the first user device to prompt a user of the first user device to indicate whether the user wants to delete the initial message content or not delete the initial message content.

16. The method of claim 15, further comprising: receiving, using one or more computers, data corresponding to an indication that the user of the first user device wants to delete the initial message content; and in response to receiving data corresponding to an indication that the user of the first user device wants to delete the initial message content, deleting, using one or more computers, the initial message, wherein deletion of the initial message content prohibits any other user from viewing the first message content after its deletion.

17. A system for identifying offensive message content comprising: one or more computers; and one or more computer-readable storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations, the operations comprising: for each particular responsive message of a plurality of responsive messages received in response to an initial message: providing, by the one or more computers, content of the particular responsive message as an input to a machine learning model that has been trained to predict a likelihood that an initial message provided by a first user device includes offensive content based on processing of responsive message content from a different user device provided in response to the initial message; processing, by the one or more computers, the content of the particular responsive message through the machine learning model to generate output data indicating a likelihood that the initial message includes offensive content; and storing, by the one or more computers, the generated output data; determining, by the one or more computers and based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content; and based on a determination, by the one or more computers, that the output data generated for each of the plurality of responsive messages indicates that the initial message likely includes offensive content, performing, by the one or more computers, one or more remedial operations to mitigate exposure to the offensive content, wherein performing the one or more remedial operations comprises: adjusting, using the one or more computers, a content score associated with the initial message content, wherein the adjusted content scores causes the initial message content to be demoted in list of content items.

18. A computer readable-store media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations, the operations comprising: for each particular responsive message of a plurality of responsive messages received in response to an initial message: providing content of the particular responsive message as an input to a machine learning model that has been trained to predict a likelihood that an initial message provided by a first user device includes offensive content based on processing of responsive message content from a different user device provided in response to the initial message; processing the content of the particular responsive message through the machine learning model to generate output data indicating a likelihood that the initial message includes offensive content; and storing the generated output data; determining, based on the output data generated for each of the plurality of responsive messages, whether the initial message likely includes offensive content; and based on a determination that the output data generated for each of the plurality of responsive messages indicates that the initial message likely includes offensive content, performing one or more remedial operations to mitigate exposure to the offensive content, wherein performing the one or more remedial operations comprises: adjusting a content score associated with the initial message content, wherein the adjusted content scores causes the initial message content to be demoted in list of content items.