A content delivery network (CDN) service provider extends a content delivery network to gather information on atomically identifiable web clients (called “user agents”) as such computer-implemented entities interact with the CDN across different domains being managed by the CDN service provider. In one embodiment, a set of machines, processes, programs, and data comprise a data system. The data system tracks user agents, preferably via cookies, although one or more passive techniques may be used. A user agent may be a cookie-able device having a cookie store. As the user agent navigates across sites, a CDN-specific unique identifier used by the system to correlate user agents is generated. Preferably, the unique identifier is stored as an encrypted cookie. The unique identifier represents one user agent (and, thus, one cookie-able device's store). The system tracks user agent behavior on and across customer sites that are served by the CDN, and these behaviors are classified into identifiable “segments” that may be used to create a profile. CDN customers use the data system to obtain information that characterizes the user agent.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. Apparatus for use in a content delivery network in which participating content providers offload given content for delivery from content servers managed by a content delivery network service provider, comprising: a processor; a computer memory holding computer program instructions that when executed by the processor perform a bot mitigation method comprising: tracking a same client machine user agent across a plurality of content provider domains that belong to the participating content providers, the plurality of content provider domains being managed by the content delivery network service provider on behalf of the participating content providers, the tracking including the following sub-steps carried out with respect to at least one content provider domain visited by the client machine user agent: redirecting the client machine user agent from the content provider domain to a content delivery network service provider namespace and setting a master cookie, automatically redirecting the client machine user agent back to the content provider domain from which the client machine user agent was redirected; and setting a content provider domain-specific cookie equal to the value of the master cookie; based on information collection during the tracking step, making a determination regarding whether the client machine user agent should be characterized as being controlled by an automated agent as opposed to a human user; if as a result of the determination the client machine user agent is characterized as being controlled by an automated agent as opposed to a human user, taking a mitigation action.
A content delivery network (CDN) system detects and mitigates automated bots. The CDN tracks a single user agent (like a web browser) across multiple websites served by the CDN. This tracking involves redirecting the user agent to a CDN-controlled namespace to set a "master cookie," then redirecting the user back to the original website, also setting a website-specific cookie matching the master cookie's value. Based on the user agent's behavior gathered during tracking, the system determines if it's a bot or a human. If identified as a bot, the system takes a mitigation action. The system includes a processor and memory with instructions for performing the bot detection and mitigation.
2. The apparatus as described in claim 1 wherein the mitigation action includes serving given content to the client machine user agent.
The bot mitigation system described above, where a mitigation action is taken when a bot is detected, includes serving specific content to the bot. The content delivered to the bot is different than content served to a normal user.
3. The apparatus as described in claim 2 wherein the given content is a dummy page.
In the bot mitigation system, the specific content delivered to a detected bot, as described above, is a dummy page. The dummy page may appear to be a real page but does not provide useful content to the bot.
4. The apparatus as described in claim 2 wherein the given content is alternative content.
In the bot mitigation system, the specific content delivered to a detected bot, as described above, is alternative content. The alternative content can be used to deceive the bot or render it ineffective.
5. The apparatus as described in claim 1 wherein the mitigation action includes providing a given response to the client machine user agent that differs from a response that would be provided if the client machine user agent were determined to be associated with a human user.
The bot mitigation system described above, where a mitigation action is taken when a bot is detected, includes providing a different response to the bot than it would provide to a human user. The bot gets a modified HTTP response, different content, or other altered data.
6. The apparatus as described in claim 5 wherein the given response is associated with a low quality of service.
In the bot mitigation system, the different response provided to a detected bot, as described above, is associated with a low quality of service. The bot may experience slower load times, incomplete data, or intermittent errors.
7. The apparatus as described in claim 1 wherein the mitigation action includes routing the client machine user agent to a subset of servers where the client machine user agent is forced to compete for resources with a plurality of other client machine user agents.
The bot mitigation system described above, where a mitigation action is taken when a bot is detected, includes routing the bot to a subset of servers where it must compete for resources with other bots. This isolates the bots and reduces their impact on legitimate users.
8. The apparatus as described in claim 1 wherein the determination is based on a value that represents a confidence of the content delivery network service provider that the client machine user agent represents a human user as opposed to an automated agent.
In the bot mitigation system described above, the determination of whether a user agent is a bot is based on a confidence score. This score represents the CDN service provider's certainty that the user agent is human rather than a bot. Higher scores indicate higher confidence in human identification.
9. The apparatus as described in claim 1 wherein the client machine user agent is a web browser.
In the bot mitigation system described above, the client machine user agent being tracked is a web browser. The system monitors and analyzes browser behavior to differentiate bots from humans.
10. The apparatus as described in claim 1 wherein the client machine user agent is associated with a cookie-able device having a cookie store.
In the bot mitigation system described above, the client machine user agent being tracked is associated with a device that supports cookies. The system uses cookies to track the user agent across different websites.
11. The apparatus as described in claim 1 further including providing the determination to a participating content provider.
The bot mitigation system described above also provides the bot determination (bot or human) to the content provider whose site the user is visiting. This allows the content provider to take its own actions based on the CDN's bot detection.
12. The apparatus as described in claim 11 wherein the mitigation action is taken by the participating content provider.
A system for managing content distribution involves a network of content providers that collaborate to detect and mitigate malicious or unauthorized content. The system identifies suspicious content by analyzing distribution patterns, user behavior, or other indicators. When such content is detected, a mitigation action is triggered to prevent its further spread. The mitigation action can include blocking, quarantining, or removing the content from distribution channels. In this particular implementation, the mitigation action is executed by the participating content provider that initially detected the suspicious content. This ensures that the response is decentralized, reducing reliance on a central authority and improving scalability. The system may also include mechanisms for verifying the legitimacy of the detection and mitigation actions to prevent false positives. The content providers may communicate with each other to share threat intelligence, allowing for coordinated responses across the network. This approach enhances security and efficiency in content distribution by leveraging the collective capabilities of participating providers.
13. The apparatus as described in claim 11 wherein the participating content provider provides limited inventory items.
In the bot mitigation system, where the bot determination is provided to the content provider, the content provider provides limited inventory items, and this action is taken to mitigate a bot. This is useful for scenarios like ticket sales, where bots try to purchase large numbers of tickets.
14. The apparatus as described in claim 13 wherein the automated agent is a ticket bot.
In the bot mitigation system where the content provider limits inventory items, as described above, the automated agent is a ticket bot. The content provider limits the number of tickets available to any single user or IP address based on the determination of bot activity.
15. The apparatus as described in claim 11 wherein the participating content provider provides a social networking service.
In the bot mitigation system, where the bot determination is provided to the content provider, the content provider provides a social networking service. The bot detection helps protect the social network from malicious activity.
16. The apparatus as described in claim 15 wherein the automated agent is a friend bot.
In the bot mitigation system where the content provider provides a social networking service, as described above, the automated agent is a friend bot. The system detects and mitigates bots that automatically add friends or spread spam.
17. The apparatus as described in claim 11 further including charging the participating content provider a fee.
The bot mitigation system described above also includes charging the participating content provider a fee for the bot detection and mitigation service. The CDN provider monetizes the bot protection service.
18. The apparatus as described in claim 1 wherein the determination is based on diversity of content provider domains visited by the client machine user agent.
In the bot mitigation system described above, the determination of whether a user agent is a bot is based on the diversity of content provider domains visited by the user agent. Bots may exhibit browsing patterns that are less diverse than those of typical human users.
19. The apparatus as described in claim 1 wherein the determination is based on purchase-to-catalog page ratio with respect to one or more pages associated with a given content provider domain.
In the bot mitigation system described above, the determination of whether a user agent is a bot is based on the purchase-to-catalog page ratio with respect to one or more pages associated with a given content provider domain. A high purchase-to-catalog view ratio might indicate bot activity.
20. The apparatus as described in claim 1 wherein the determination is based on an amount of time that has passed from a last browsing session initiated by the client machine user agent or an amount of time that the client machine user agent has been online during a current browsing session.
In the bot mitigation system described above, the determination of whether a user agent is a bot is based on the time that has passed from the last browsing session initiated by the user agent or the amount of time that the user agent has been online during a current browsing session. Unusual session timing could indicate automated behavior.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
August 17, 2007
July 9, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.