Device, method, and graphical user interface for manipulating user interfaces based on fingerprint sensor inputs

This application is a continuation of U.S. patent application Ser. No. 17/525,445, “DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR MANIPULATING USER INTERFACES BASED ON FINGERPRINT SENSOR INPUTS,” filed on Nov. 12, 2021, which is a continuation of U.S. patent application Ser. No. 14/402,087, now U.S. Pat. No. 11,209,961, filed on Nov. 18, 2014, “DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR MANIPULATING USER INTERFACES BASED ON FINGERPRINT SENSOR INPUTS,” which is a national phase application under 35 U.S.C. § 371 of International Application No. PCT/US2013/041868, “Device, Method, and Graphical User Interface for Manipulating User Interfaces Based on Fingerprint Sensor Inputs.” filed May 20, 2013, which claims priority to U.S. Provisional Patent Application Ser. No. 61/794,710, filed on Mar. 15, 2013, entitled “Device, Method, and Graphical User Interface for Manipulating User Interfaces Based on Fingerprint Sensor Inputs;” U.S. Provisional Patent Application No. 61/667,926, filed Jul. 3, 2012, entitled “Device, Method, and Graphical User Interface for Manipulating User Interfaces Based on Fingerprint Sensor Inputs;” and U.S. Provisional Patent Application No. 61/649,207, filed May 18, 2012, entitled “Device, Method, and Graphical User Interface for Manipulating User Interfaces Based on Fingerprint Sensor Inputs,” which applications are incorporated by reference herein in their entirety.

This relates generally to electronic devices with fingerprint sensors, including but not limited to electronic devices with fingerprint sensors that detect inputs for manipulating user interfaces.

The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a display. Additionally, some electronic devices include fingerprint sensors for authenticating users.

Exemplary manipulations include adjusting the position and/or size of one or more user interface objects or activating buttons or opening files/applications represented by user interface objects, as well as associating metadata with one or more user interface objects or otherwise manipulating user interfaces. Exemplary user interface objects include digital images, video, text, icons, control elements such as buttons and other graphics. A user will, in some circumstances, need to perform such manipulations on user interface objects in a file management program (e.g., Finder from Apple Inc. of Cupertino, California), an image management application (e.g., Aperture or iPhoto from Apple Inc. of Cupertino, California), a digital content (e.g., videos and music) management application (e.g., iTunes from Apple Inc. of Cupertino, California), a drawing application, a presentation application (e.g., Keynote from Apple Inc. of Cupertino, California), a word processing application (e.g., Pages from Apple Inc. of Cupertino, California), a website creation application (e.g., iWeb from Apple Inc. of Cupertino, California), a disk authoring application (e.g., iDVD from Apple Inc. of Cupertino, California), or a spreadsheet application (e.g., Numbers from Apple Inc. of Cupertino, California).

But methods for performing these manipulations are cumbersome and inefficient. In addition, these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices.

Accordingly, there is a need for electronic devices with faster, more efficient methods and interfaces for manipulating user interfaces. Such methods and interfaces optionally complement or replace conventional methods for manipulating user interfaces. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

The above deficiencies and other problems associated with user interfaces for electronic devices with touch-sensitive surfaces are reduced or eliminated by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a “touch screen” or “touch screen display”). In some embodiments, the device has a fingerprint sensor. In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through finger contacts and gestures on the touch-sensitive surface and/or the fingerprint sensor. In some embodiments, the functions optionally include image editing, drawing, presenting, word processing, website creating, disk authoring, spreadsheet making, game playing, telephoning, video conferencing, e-mailing, instant messaging, workout support, digital photographing, digital videoing, web browsing, digital music playing, and/or digital video playing. Executable instructions for performing these functions are, optionally, included in a non-transitory computer readable storage medium or other computer program product configured for execution by one or more processors.

In accordance with some embodiments, an electronic device includes a fingerprint sensor, a display, and/or a touch-sensitive surface, one or more processors, memory, and one or more programs; the one or more programs are stored in the memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing the operations of any of the methods described above. In accordance with some embodiments, a graphical user interface on an electronic device with a fingerprint sensor, a display, optionally a touch-sensitive surface, a memory, and one or more processors to execute one or more programs stored in the memory includes one or more of the elements displayed in any of the methods described above, which are updated in response to inputs, as described in any of the methods described above. In accordance with some embodiments, a computer readable storage medium has stored therein instructions which when executed by an electronic device with a fingerprint sensor and optionally a display and/or, a touch-sensitive surface, cause the device to perform the operations of any of the methods described above. In accordance with some embodiments, an electronic device includes: a fingerprint sensor and optionally, a display and/one or a touch-sensitive surface; and means for performing the operations of any of the methods described above. In accordance with some embodiments, an information processing apparatus, for use in an electronic device with a fingerprint sensor and optionally a display and/or a touch-sensitive surface, includes means for performing the operations of any of the methods described above.

There is a need for electronic devices with faster, more efficient methods and interfaces for conveniently interacting with user interfaces (e.g., for navigating through an application, for switching from one application interface to another or for displaying a multi-tasking user interface) on a portable multi-function device. Such methods and interfaces may complement or replace conventional methods for interacting with user interfaces (e.g., for navigating through an application user interface, switching between application user interfaces or for displaying a multi-tasking user interface). Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a fingerprint sensor. The method includes: displaying, on the display, a first user interface. The method further includes, while displaying the first user interface on the display, detecting movement of a fingerprint on the fingerprint sensor. The method also includes in response to detecting movement of the fingerprint on the fingerprint sensor: in accordance with a determination that the movement of the fingerprint is in a first direction, navigating through the first user interface; and in accordance with a determination that the movement of the fingerprint is in a second direction different from the first direction, displaying a second user interface different from the first user interface on the display.

In accordance with some embodiments, an electronic device includes a display unit configured to display a first user interface, a fingerprint sensor unit; and a processing unit coupled to the display unit and the fingerprint sensor unit. In some embodiments, the processing unit includes a detecting unit, a navigating unit, a display enabling unit, and a replacing unit. The processing unit is configured to: while displaying the first user interface on the display, detect movement of a fingerprint on the fingerprint sensor; and in response to detecting movement of the fingerprint on the fingerprint sensor: in accordance with a determination that the movement of the fingerprint is in a first direction, navigate through the first user interface; and in accordance with a determination that the movement of the fingerprint is in a second direction different from the first direction, enable display of a second user interface different from the first user interface on the display unit.

Thus, electronic devices with displays, and fingerprint sensors are provided with faster, more efficient methods and interfaces for switching between user interfaces, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for switching between user interfaces.

There is a need for electronic devices with faster, more efficient methods and interfaces having buttons that serve multiple purposes (e.g., allowing the button, upon activation, to cause the device to perform a first operation or, alternatively, a second operation different from the first, depending on certain criteria). Such methods and interfaces may complement or replace conventional methods for allowing buttons to serve multiple purposes. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges. For portable multifunction devices, such methods and interfaces also conserve valuable physical space on housing of the device, allowing for a larger screen and better user experience.

In accordance with some embodiments, a method is performed at an electronic device with a button including an integrated fingerprint sensor. The method includes: detecting a fingerprint on the integrated fingerprint sensor. The method further includes, while continuing to detect the fingerprint on the integrated fingerprint sensor, detecting activation of the button. The method further includes, in response to detecting activation of the button and in accordance with a determination that the fingerprint was not continuously detected on the integrated fingerprint sensor for more than a predetermined period of time prior to detecting activation of the button, performing a first operation, where the first operation is a predefined operation associated with activation of the button. The method further includes, in response to detecting activation of the button and in accordance with a determination that the fingerprint was continuously detected on the integrated fingerprint sensor for more than the predetermined period of time prior to detecting activation of the button, performing a second operation different from the first operation, where the second operation is an operation associated with a user interface displayed on the display immediately prior to detecting activation of the button.

In accordance with some embodiments, an electronic device includes a button unit with an integrated fingerprint sensor unit; and a processing unit coupled to the button unit. The processing unit is configured to: detect a fingerprint on the integrated fingerprint sensor unit. The processing unit is further configured to: while continuing to detect the fingerprint on the integrated fingerprint sensor unit, detect activation of the button unit. The processing unit is further configured to: in response to detecting activation of the button unit and in accordance with a determination that the fingerprint was not continuously detected on the integrated fingerprint sensor unit for more than a predetermined period of time prior to detecting activation of the button unit, perform a first operation, wherein the first operation is a predefined operation associated with activation of the button unit. The processing unit is further configured to: in response to detecting activation of the button unit and in accordance with a determination that the fingerprint was continuously detected on the integrated fingerprint sensor unit for more than the predetermined period of time prior to detecting activation of the button unit, perform a second operation different from the first operation, wherein the second operation is an operation associated with a user interface displayed on the display unit immediately prior to detecting activation of the button unit.

Thus, electronic devices with buttons with integrated fingerprint sensors are provided with faster, more efficient methods and interfaces for allowing said buttons to serve multiple purposes, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for allowing buttons to serve multiple purposes.

There is a need for electronic devices with faster, more efficient methods and interfaces for locking an orientation of a user interface. Such methods and interfaces may complement or replace conventional methods for locking an orientation of a user interface. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a fingerprint sensor. The method includes: displaying a user interface on the display, where the user interface has a first orientation-specific mode of operation associated with a first orientation of the device and a second orientation-specific mode of operation that is associated with a second orientation of the device that is different from the first orientation of the device; while the orientation of the user interface is not locked, displaying the user interface in the first orientation-specific mode of operation when the device is in the first orientation, and displaying the user interface in the second orientation-specific mode of operation when the device is in the second orientation; detecting a fingerprint on the fingerprint sensor; while the user interface is in the first orientation-specific mode of operation, detecting rotation of the fingerprint on the fingerprint sensor in a first direction; in response to detecting rotation of the fingerprint on the fingerprint sensor in the first direction, locking the user interface in the first orientation-specific mode of operation; and while the user interface is locked in the first orientation-specific mode of operation, detecting that the device is in the second orientation and maintaining the user interface in the first orientation-specific mode of operation.

In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface, where the user interface has a first orientation-specific mode of operation associated with a first orientation of the device and a second orientation-specific mode of operation that is associated with a second orientation of the device that is different from the first orientation of the device, a fingerprint sensor unit; and a processing unit coupled to the display unit and the fingerprint sensor unit. The processing unit is configured to: while the orientation of the user interface is not locked: enable display of the user interface in the first orientation-specific mode of operation when the device is in the first orientation, and enable display of the user interface in the second orientation-specific mode of operation when the device is in the second orientation; detect a fingerprint on the fingerprint sensor unit; while the user interface is in the first orientation-specific mode of operation, detect rotation of the fingerprint on the fingerprint sensor unit in a first direction; in response to detecting rotation of the fingerprint on the fingerprint sensor unit in the first direction, lock the user interface in the first orientation-specific mode of operation; and while the user interface is locked in the first orientation-specific mode of operation, detect that the device is in the second orientation and maintain the user interface in the first orientation-specific mode of operation.

Thus, electronic devices with displays and fingerprint sensors are provided with faster, more efficient methods and interfaces for locking an orientation of user interface, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for locking an orientation of user interface.

There is a need for electronic devices with faster, more efficient methods and interfaces for controlling noise reduction in recorded audio with a fingerprint sensor. Such methods and interfaces may complement or replace conventional methods for controlling noise reduction in recorded audio with a fingerprint sensor. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a fingerprint sensor. The method includes recording media with the device, where recording the media includes recording audio. The method further includes, while recording the media, reducing, in the recorded audio, noise occurring on a first side of the device, detecting a fingerprint on the fingerprint sensor and, in response to detecting the fingerprint on the fingerprint sensor, ceasing to reduce noise occurring on the first side of the device.

In accordance with some embodiments, an electronic device includes a fingerprint sensor unit configured to detect fingerprints and a processing unit coupled to the fingerprint sensor unit. The processing unit is configured to record media with the device, where recording the media includes recording audio. The processing unit is further configured to, while recording the media, reduce, in the recorded audio, noise occurring on a first side of the device, detect a fingerprint on the fingerprint sensor unit, and, in response to detecting the fingerprint on the fingerprint sensor unit, cease to reduce noise occurring on the first side of the device.

Thus, electronic devices with fingerprint sensors are provided with faster, more efficient methods and interfaces for controlling noise reduction in recorded audio with a fingerprint sensor, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for controlling noise reduction in recorded audio with a fingerprint sensor.

There is a need for electronic devices with faster, more efficient methods and interfaces for performing operations that are adapted for a current user of the device. Such methods and interfaces may complement or replace conventional methods that are adapted for a current user of the device. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display, a respective input element, and a fingerprint sensor. The method includes: obtaining a plurality of profiles, where the plurality of profiles includes one or more user-specific profiles and one or more class-based profiles, distinct from the user-specific profiles, that each correspond to a respective class of users; detecting a first fingerprint on the fingerprint sensor; selecting one or more class-based profiles associated with the device as active profiles that correspond to the first fingerprint without regard to authentication of the first fingerprint as a previously registered fingerprint, including, in accordance with a determination that the first fingerprint corresponds to a fingerprint of a user in a first class of users that are associated with a first class-based profile, selecting the first class-based profile as an active profile; receiving an input with the respective input element, wherein the input corresponds to a request to perform one or more operations at the device; and in response to receiving the input with the respective input element while the first class-based profile is selected as an active profile, performing a respective operation based on the request and the first class-based profile.

In accordance with some embodiments, an electronic device includes a display unit, a respective input element unit, a fingerprint sensor unit, and a processing unit coupled to the display unit, the respective input element unit, and the fingerprint sensor unit. In some embodiments, the processing unit includes an obtaining unit, a detecting unit, a selecting unit, a performing unit, a determining unit, and an identifying unit. The processing unit is configured to: obtain a plurality of profiles, where the plurality of profiles includes one or more user-specific profiles and one or more class-based profiles, distinct from the user-specific profiles, that each correspond to a respective class of users; detect a first fingerprint on the fingerprint sensor unit; select one or more class-based profiles associated with the device as active profiles that correspond to the first fingerprint without regard to authentication of the first fingerprint as a previously registered fingerprint, including, in accordance with a determination that the first fingerprint corresponds to a fingerprint of a user in a first class of users that are associated with a first class-based profile, select the first class-based profile as an active profile; receive an input with the respective input element unit, where the input corresponds to a request to perform one or more operations at the device; and in response to receiving the input with the respective input element unit while the first class-based profile is selected as an active profile, perform a respective operation based on the request and the first class-based profile.

Thus, electronic devices with displays, respective input elements, and fingerprint sensors are provided with faster, more efficient methods and interfaces for performing operations that are adapted for a current user of the device, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for performing operations based on a class-based profile.

There is a need for electronic devices with faster, more efficient methods and interfaces for changing logging preferences based on fingerprints. Such methods and interfaces may complement or replace conventional methods for changing logging preferences. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges. For portable multifunction devices, such methods and interfaces allow lost or stolen devices to be more easily recovered.

In accordance with some embodiments, a method is performed at an electronic device with a display, a respective input element and a fingerprint sensor. The method includes: detecting a first fingerprint on the fingerprint sensor and determining whether the first fingerprint is associated with a user profile associated with the device. The method further includes, after determining whether the first fingerprint is associated with a user profile associated with the device and in accordance with a determination that the first fingerprint is associated with a respective user profile associated with the device, setting the respective user profile as an active profile. The method further includes, after determining whether the first fingerprint is associated with a user profile associated with the device and in accordance with a determination that the first fingerprint is not associated with any user profile associated with the device, setting a default profile as an active profile. The method further includes, receiving an input with the respective input element, where the input corresponds to a request to perform a requested operation. The method further includes, in response to receiving the input with the respective input element and in accordance with a determination that the respective user profile is the active profile, performing a first set of one or more operations associated with the requested operation in accordance with logging preferences of the respective user profile. The method further includes, in response to receiving the input with the respective input element and in accordance with a determination that the respective user profile is the active profile, in accordance with a determination that the default profile is the active profile, performing a second set of operations associated with the requested operation in accordance with logging preferences of the default profile.

In accordance with some embodiments, an electronic device includes a respective input unit configured to receive an input, wherein the input corresponds to a request to perform a requested operation; a fingerprint sensor unit; and a processing unit coupled to the fingerprint sensor unit and the respective input unit, the processing unit configured to: detect a first fingerprint on the fingerprint sensor unit and determine whether the first fingerprint is associated with a user profile associated with the device. The processing unit is further configured to, after determining whether the first fingerprint is associated with a user profile associated with the device: in accordance with a determination that the first fingerprint is associated with a respective user profile associated with the device, set the respective user profile as an active profile; and in accordance with a determination that the first fingerprint is not associated with any user profile associated with the device, set a default profile as an active profile. The processing unit is further configured to, in response to receiving the input with the respective input unit: in accordance with a determination that the respective user profile is the active profile, perform a first set of one or more operations associated with the requested operation in accordance with logging preferences of the respective user profile; and in accordance with a determination that the default profile is the active profile, perform a second set of operations associated with the requested operation in accordance with logging preferences of the default profile.

Thus, electronic devices with input elements and fingerprint sensors are provided with faster, more efficient methods and interfaces for changing logging preferences based on fingerprints, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for changing logging preferences.

There is a need for electronic devices with faster, more efficient methods and interfaces for performing a respective restricted operation based on fingerprint-verification information (e.g., when the fingerprint-verification information meets respective authorization criteria for the respective restricted operation). Such methods and interfaces may complement or replace conventional methods for performing a respective restricted operation based. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method includes obtaining fingerprint-verification criteria for controlling verification of fingerprints. For each of a plurality of respective inputs that correspond to respective restricted operations, the method further includes identifying fingerprint features of the respective input and determining fingerprint-verification information for the respective input based at least in part on: a comparison between the fingerprint features of the respective input and fingerprint features of one or more previously registered fingerprints; and the fingerprint-verification criteria. Finally, in response to detecting the respective input, the method includes: in accordance with a determination that the fingerprint-verification information meets respective authorization criteria for the respective restricted operation, performing the respective restricted operation; and in accordance with a determination that the fingerprint-verification information does not meet the respective authorization criteria for the restricted operation, forgoing performing the respective restricted operation.

In accordance with some embodiments, an electronic device includes a display unit configured to display information; a fingerprint sensor unit; and a processing unit coupled to the display unit and the fingerprint sensor unit. The processing unit is configured to obtain fingerprint-verification criteria for controlling verification of fingerprints. For each of a plurality of respective inputs that correspond to respective restricted operations, the processing unit is further configured to identify fingerprint features of the respective input and determine fingerprint-verification information for the respective input based at least in part on: a comparison between the fingerprint features of the respective input and fingerprint features of one or more previously registered fingerprints; and the fingerprint-verification criteria. Finally, in response to detecting the respective input, the processing unit is configured to: in accordance with a determination that the fingerprint-verification information meets respective authorization criteria for the respective restricted operation, perform the respective restricted operation; and in accordance with a determination that the fingerprint-verification information does not meet the respective authorization criteria for the restricted operation, forgo performance of the respective restricted operation.

Thus, electronic devices with displays and fingerprint sensors are provided with faster, more efficient methods and interfaces for performing a respective restricted operation based on fingerprint-verification information, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for performing a respective restricted operation.

There is a need for electronic devices with faster, more efficient methods and interfaces for associating identity and location information. Such methods and interfaces may complement or replace conventional methods for associating identity and location information. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a fingerprint sensor. The method includes displaying a map on the display and detecting an activation input. The activation input is associated with a displayed location on the map that corresponds to a respective geographic location, and a fingerprint detected on the fingerprint sensor. The method further includes, in response to detecting the activation input, identifying a user that corresponds to the fingerprint, and performing an operation associated with the respective geographic location and the user.

In accordance with some embodiments, an electronic device includes a display unit configured display a map; a fingerprint sensor unit configured to detect a fingerprint and a processing unit coupled to the display unit and the fingerprint sensor unit. In some embodiments, the display unit is a touch-sensitive display unit and the fingerprint sensor unit detects fingerprints on the touch-sensitive display unit. The processing unit is configured to: detect an activation input, the activation input being associated with: a displayed location on the map that corresponds to a respective geographic location and a fingerprint detected on the fingerprint sensor unit; and, in response to detecting the activation input: identify a user that corresponds to the fingerprint, and perform an operation associated with the respective geographic location and the user.

Thus, electronic devices with displays and fingerprint sensors are provided with faster, more efficient methods and interfaces for associating identity and location information, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for associating identity and location information.

There is a need for electronic devices with faster, more efficient methods and interfaces for enabling one or more users to create and define their own personal workspace or work area. Accordingly, the disclosed embodiments provide methods for assigning respective workspaces to one or more users based on a detection of the users' fingerprints in the contacts that the user places on the touch-sensitive surface to define the bounds of a respective workspace within a bigger shared workspace on a multifunction device, and to associate their respective workspace with their respective custom user-settings based on their individual preferences. Such methods and interfaces may complement or replace conventional methods for assigning workspaces to one or more users. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display, a touch-sensitive surface, and a fingerprint sensor. The method includes: detecting a first set of one or more inputs on the touch-sensitive surface that define a first area of the touch-sensitive surface. The method further includes, after detecting the first set of one or more inputs: determining that the first set of one or more inputs includes a first contact that includes a first fingerprint associated with a first user; and associating the first area of the touch-sensitive surface with the first user based at least in part on the determination that the first set of one or more inputs includes the first contact that includes the first fingerprint associated with the first user.

In accordance with some embodiments, an electronic device includes a display unit configured to display one or more user interface objects, a touch-sensitive surface unit configured to receive a contact on the touch-sensitive surface unit, a fingerprint sensor unit; and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the fingerprint sensor unit. The processing unit is configured to: detect a first set of one or more inputs on the touch-sensitive surface unit that define a first area of the touch-sensitive surface unit; and after detecting the first set of one or more inputs: determine that the first set of one or more inputs includes a first contact that includes a first fingerprint associated with a first user; and associate the first area of the touch-sensitive surface unit with the first user based at least in part on the determination that the first set of one or more inputs includes the first contact that includes the first fingerprint associated with the first user.

Thus, electronic devices with displays, touch-sensitive surfaces, and fingerprint sensors are provided with faster, more efficient methods and interfaces for associating areas of a touch-sensitive surface with one or more users, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for assigning workspaces to one or more users.

There is a need for electronic devices with faster, more efficient methods and interfaces for training a user on proper fingering techniques used, for example, when typing on a computer keyboard or depressing a key on a musical keyboard (e.g., playing a piano). Such methods and interfaces may complement or replace conventional methods for training a user on proper fingering techniques. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a touch-sensitive surface. The method includes: displaying a user interface including a respective activatable user interface object and detecting a fingerprint pattern of a contact on the touch-sensitive surface that corresponds to a focus selector that is over the respective activatable user interface object on the display. The method also includes, in response to detecting the fingerprint pattern of the contact on the touch-sensitive surface that corresponds to the focus selector that is over the respective activatable user interface object on the display: analyzing the fingerprint pattern of the contact on the touch-sensitive surface to determine one or more fingering characteristics of the contact and determining whether the contact meets predefined preferred fingering criteria based on the one or more fingering characteristics. The method further includes, in accordance with a determination that the contact does not meet the preferred fingering criteria, providing feedback indicating that the preferred fingering criteria have not been met and, in accordance with a determination that the contact meets the preferred fingering criteria, performing an operation associated with the activatable user interface object.

In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to receive user contacts, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit configured to enable display of a user interface including a respective activatable user interface object and to detect a fingerprint pattern of a contact on the touch-sensitive surface unit that corresponds to a focus selector that is over the respective activatable user interface object on the display unit. The processing unit is also configured to, in response to detecting the fingerprint pattern of the contact on the touch-sensitive surface unit that corresponds to the focus selector that is over the respective activatable user interface object on the display unit, analyze the fingerprint pattern of the contact on the touch-sensitive surface unit to determine one or more fingering characteristics of the contact and determine whether the contact meets predefined preferred fingering criteria based on the one or more fingering characteristics. The processing unit is further configured to, in accordance with a determination that the contact does not meet the preferred fingering criteria, provide feedback indicating that the preferred fingering criteria have not been met and, in accordance with a determination that the contact meets the preferred fingering criteria, perform an operation associated with the activatable user interface object.

There is a need for electronic devices with faster, more efficient methods and interfaces for operating a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation. Such methods and interfaces may complement or replace conventional methods for operating a touch-sensitive surface. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a touch-sensitive surface. The method includes: detecting a contact in a first region of the touch-sensitive surface, where the first region of the touch-sensitive surface is in a reduced-sensitivity mode of operation; in response to detecting the contact: determining whether the contact meets one or more fingerprint-sensor activation criteria; in accordance with a determination that the contact meets the fingerprint-sensor activation criteria, operating the first region of the touch-sensitive surface in an enhanced-sensitivity mode of operation; and in accordance with a determination that the contact does not meet the fingerprint-sensor activation criteria, continuing to operate the first region of the touch-sensitive surface in the reduced-sensitivity mode of operation.

In accordance with some embodiments, an electronic device includes a display unit, a touch-sensitive surface unit configured to receive contacts, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: detect a contact in a first region of the touch-sensitive surface unit, where the first region of the touch-sensitive surface unit is in a reduced-sensitivity mode of operation; in response to detecting the contact: determine whether the contact meets fingerprint-sensor activation criteria; in accordance with a determination that the contact meets the fingerprint-sensor activation criteria, operate the first region of the touch-sensitive surface unit in an enhanced-sensitivity mode of operation; and in accordance with a determination that the contact does not meet the fingerprint-sensor activation criteria, continue to operate the first region of the touch-sensitive surface unit in the reduced-sensitivity mode of operation

Thus, electronic devices with displays and touch-sensitive surfaces are provided with faster, more efficient methods and interfaces for operating a touch-sensitive surface by operating a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for operating a touch-sensitive surface.

There is a need for electronic devices with faster, more efficient methods and interfaces for performing specific operations on devices. Such methods and interfaces may complement or replace conventional methods for performing specific operations on devices. Such methods and interfaces reduce the cognitive burden on a user and produce a morc efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a touch-sensitive surface. The method includes: associating a first fingerprint with a first operation; displaying, on the display, a respective user interface that includes affordances for performing a plurality of operations other than the first operation; while displaying the respective user interface that includes affordances for performing the plurality of operations other than the first operation, detecting a first gesture that includes detecting the first fingerprint on the touch-sensitive surface; and in response to detecting the first gesture, performing the first operation.

In accordance with some embodiments, an electronic device includes a display unit configured to display a respective user interface that includes affordances for performing a plurality of operations other than a first operation, a touch-sensitive surface unit configured to receive gestures, and a processing unit coupled to the display unit and the touch-sensitive surface unit. The processing unit is configured to: associate a first fingerprint with the first operation; while enabling display of the respective user interface that includes affordances for performing the plurality of operations other than the first operation, detect a first gesture that includes detecting the first fingerprint on the touch-sensitive surface unit; and in response to detecting the first gesture, perform the first operation

Thus, electronic devices with displays touch-sensitive surfaces and fingerprint sensors are provided with faster, more efficient methods and interfaces for performing operations associated with fingerprint gestures, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for performing operations associated with fingerprint gestures.

There is a need for electronic devices with faster, more efficient methods and interfaces for displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control. Such methods and interfaces may complement or replace conventional methods for displaying a respective control for a user interface. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display, a touch-sensitive surface and a fingerprint sensor. The method includes displaying a user interface on the display. The method further includes detecting a first fingerprint associated with moving a respective control for the user interface on the touch-sensitive surface, where the first fingerprint corresponds to a focus selector at a first location on the display that does not include the respective control. In response to detecting the first fingerprint, the method includes displaying the respective control at the first location on the display.

In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface; a touch-sensitive surface unit configured to receive one or more contacts; a fingerprint sensor unit; and a processing unit coupled to the display unit, the touch-sensitive surface unit and the fingerprint sensor unit. The processing unit is configured to: detect a first fingerprint associated with moving a respective control for the user interface on the touch-sensitive surface unit, where the first fingerprint corresponds to a focus selector at a first location on the display unit that does not include the respective control; and in response to detecting the first fingerprint, enable display of the respective control at the first location on the display unit.

Thus, electronic devices with displays, touch-sensitive surfaces and fingerprint sensors are provided with faster, more efficient methods and interfaces for displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for displaying a respective control for a user interface.

There is a need for electronic devices with faster, more efficient methods and interfaces for configuring a second electronic device as an auxiliary display of a first electronic device. Such methods and interfaces may complement or replace conventional methods for configuring a second electronic device as an auxiliary display. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a fingerprint sensor. The method includes: detecting a first fingerprint on the fingerprint sensor of the first electronic device at a first time, where the first fingerprint is registered to a respective user. The method also includes receiving a communication from a second electronic device, distinct from the first electronic device, that includes information indicating that a second fingerprint was detected on a fingerprint sensor of the second device at a second time. The method further includes, in response to detecting the first fingerprint and receiving the communication from the second electronic device: determining whether predefined auxiliary-display criteria have been met, where the auxiliary-display criteria include a criterion that is met when the second fingerprint is registered to the respective user. The method further includes, in accordance with a determination that the auxiliary-display criteria are met, sending a response to the second electronic device that enables the second electronic device to be configured as an auxiliary display for the first electronic device; and in accordance with a determination that the auxiliary display criteria are not met, foregoing sending the response to the second electronic device that enables the second electronic device to be configured as the auxiliary display for the first electronic device.

In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface, a fingerprint sensor unit, and a processing unit coupled to the display unit and the fingerprint sensor unit. The processing unit is configured to detect a first fingerprint on the fingerprint sensor unit of the first electronic device at a first time, where the first fingerprint is registered to a respective user. The processing unit is also configured to receive a communication from a second electronic device, distinct from the first electronic device, that includes information indicating that a second fingerprint was detected on a fingerprint sensor unit of the second device at a second time. The processing unit is further configured, in response to detecting the first fingerprint and receiving the communication from the second electronic device, to determine whether predefined auxiliary-display criteria have been met, where the auxiliary-display criteria include a criterion that is met when the second fingerprint is registered to the respective user. The processing unit is further configured to, in accordance with a determination that the auxiliary-display criteria are met, send a response to the second electronic device that enables the second electronic device to be configured as an auxiliary display for the first electronic device; and in accordance with a determination that the auxiliary-display criteria are not met, forego sending the response to the second electronic device that enables the second electronic device to be configured as the auxiliary display for the first electronic device.

Thus, electronic devices with displays, and fingerprint sensors are provided with faster, more efficient methods and interfaces for configuring a second electronic device as an auxiliary display of a first electronic device, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for configuring a second electronic device as an auxiliary display.

There is a need for electronic devices with faster, more efficient methods and interfaces for changing beamforming parameters based on fingerprint orientation. Such methods and interfaces may complement or replace conventional methods for changing beamforming parameters. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is performed at an electronic device with a display and a fingerprint sensor. While a first fingerprint is detected in a first orientation on the fingerprint sensor, the method includes operating a set of one or more audio elements in accordance with a first set of beamforming parameters. The method further includes: detecting a subsequent fingerprint having a second orientation different from the first orientation on the fingerprint sensor, where the subsequent fingerprint is selected from a set consisting of the first fingerprint with a changed orientation and a second fingerprint distinct from the first fingerprint; and in response to detecting the subsequent fingerprint having the second orientation on the fingerprint sensor, operating the set of one or more audio elements in accordance with a second set of beamforming parameters different from the first set of beamforming parameters.

In accordance with some embodiments, an electronic device includes a display unit configured to display information; a fingerprint sensor; a set of one or more audio units configured to generate and/or capture audio signals; and a processing unit coupled to the display unit, the fingerprint sensor unit and the set of one or more audio units. While a first fingerprint is detected in a first orientation on the fingerprint sensor unit, the processing unit is configured to operate the set of one or more audio units in accordance with a first set of beamforming parameters. The processing unit is further configured to: detect a subsequent fingerprint having a second orientation different from the first orientation on the fingerprint sensor unit, where the subsequent fingerprint is selected from a set consisting of the first fingerprint with a changed orientation and a second fingerprint distinct from the first fingerprint; and in response to detecting the subsequent fingerprint having the second orientation on the fingerprint sensor unit, operate the set of one or more audio units in accordance with a second set of beamforming parameters different from the first set of beamforming parameters.

Thus, electronic devices with displays and fingerprint sensors are provided with faster, more efficient methods and interfaces for changing beamforming parameters based on fingerprint orientation, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for changing beamforming parameters.

The methods, devices and GUIs described herein respond to inputs on a fingerprint sensor instead of, or in addition to, inputs on a touch-sensitive surface or other input device. In some implementations, a touch-sensitive surface with a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges is used as a fingerprint sensor. When a fingerprint sensor is used without a separate touch-sensitive surface, the fingerprint sensor can serve as a substitute for many of the functions of the touch-sensitive surface with a much smaller form factor, as the fingerprint sensor can detect movement of a contact over the fingerprint sensor even when the fingerprint has an area that is as large as or larger than the area of the fingerprint sensor. When a fingerprint sensor is used in addition to a separate touch-sensitive surface, the fingerprint sensor can augment the touch-sensitive surface by providing accurate detection of twisting motions of a contact, identifying different fingerprints of fingers that are used to perform gestures on the fingerprint sensor, and identifying a current user of the device. Additionally, when a fingerprint sensor is used in addition to a separate touchscreen display, the fingerprint sensor can detect touch inputs in situations where it is advantageous to avoid having fingers obscuring portions of the display (e.g., while viewing a map, a video or a game). When the touch-sensitive surface is used as a fingerprint sensor, the touch-sensitive surface optionally has spatial resolution settings that can be defined so as to switch the touch-sensitive surface (or regions of the touch-sensitive surface) between a low-resolution mode and a high-resolution mode automatically, without user intervention. In many situations the low-resolution mode consumes less power than the high-resolution mode. An advantage of operating the touch-sensitive surface in a low-resolution mode when fingerprint detection is not needed and switching the touch-sensitive surface, or a region of the touch-sensitive surface, to high-resolution mode on an as-needed basis is that such an approach conserves power while still providing high-resolution fingerprint feature sensing as-needed to improve the user experience of using the device. In implementations where the touch-sensitive surface is used as a fingerprint sensor, the term “fingerprint sensor” is used to refer to the touch-sensitive surface, or a region of the touch-sensitive surface, that is currently in high-resolution mode.

5 5 FIGS.A-N 6 6 FIGS.A-C 5 5 FIGS.A-N 6 6 FIGS.A-C Below,illustrate exemplary user interfaces for interacting with user interfaces in accordance with movement of a finger on a fingerprint sensor.are flow diagrams illustrating a method of interacting with user interfaces in accordance with movement of a finger on a fingerprint sensor. The user interfaces inare used to illustrate the processes in. 8 8 FIGS.A-O 9 9 FIGS.A-C 8 8 FIGS.A-O 9 9 FIGS.A-C Below,illustrate exemplary user interfaces for allowing buttons to serve multiple purposes.are flow diagrams illustrating a method of allowing buttons to serve multiple purposes. The user interfaces inare used to illustrate the processes in. 11 11 FIGS.A-F 12 12 FIGS.A-B 11 11 FIGS.A-F 12 12 FIGS.A-B Below,illustrate exemplary user interfaces for locking an orientation of user interface.are flow diagrams illustrating a method of locking an orientation of user interface. The user interfaces inare used to illustrate the processes in. 14 14 FIGS.A-D 15 FIG. 14 14 FIGS.A-D 15 FIG. Below,illustrate exemplary user interfaces for controlling noise reduction in recorded audio with a fingerprint sensor.is a flow diagram illustrating a method of controlling noise reduction in recorded audio with a fingerprint sensor. The user interfaces inare used to illustrate the processes in. 17 17 FIGS.A-K 18 18 FIGS.A-B 17 17 FIGS.A-K 18 18 FIGS.A-B Below,illustrate exemplary user interfaces for performing operations based on a class-based profile.are flow diagrams illustrating a method of performing operations based on a class-based profile. The user interfaces inare used to illustrate the processes in. 20 20 FIGS.A-F 21 21 FIGS.A-B 20 20 FIGS.A-F 21 21 FIGS.A-B Below,illustrate exemplary user interfaces implemented on devices that change logging preferences based on fingerprints.are flow diagrams illustrating a method of changing logging preferences based on fingerprints. The user interfaces inare used to illustrate the processes in. 23 23 FIGS.A-G 24 24 FIGS.A-D 23 23 FIGS.A-G 24 24 FIGS.A-D Below,illustrate exemplary user interfaces for performing a respective restricted operation based on fingerprint-verification information.are flow diagrams illustrating a method of performing a respective restricted operation based on fingerprint-verification information. The user interfaces inare used to illustrate the processes in. 26 26 FIGS.A-G 27 FIG. 26 26 FIGS.A-G 27 FIG. Below,illustrate exemplary user interfaces for associating identity and location information.is a flow diagram illustrating a method of associating identity and location information. The user interfaces inare used to illustrate the processes in. 29 29 FIGS.A-S 30 30 FIGS.A-B 29 29 FIGS.A-S 30 30 FIGS.A-B Below,illustrate exemplary user interfaces for associating areas of a touch-sensitive surface with one or more users.are flow diagrams illustrating a method of associating areas of a touch-sensitive surface with one or more users. The user interfaces inare used to illustrate the processes in. 32 32 FIGS.A-S 33 33 FIGS.A-C 32 32 FIGS.A-S 33 33 FIGS.A-C Below,illustrate exemplary user interfaces for training a user on proper fingering techniques.are flow diagrams illustrating a method of training a user on proper fingering techniques. The user interfaces inare used to illustrate the processes in. 35 35 FIGS.A-J 36 36 FIGS.A-B 35 35 FIGS.A-J 36 36 FIGS.A-B Below,illustrate exemplary user interfaces for operating a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation.are flow diagrams illustrating a method of operating a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation. The user interfaces inare used to illustrate the processes in. 38 38 FIGS.A-P 39 39 FIGS.A-E 38 38 FIGS.A-P 39 39 FIGS.A-E Below,illustrate exemplary user interfaces for performing operations associated with fingerprint gestures.are flow diagrams illustrating a method of performing operations associated with fingerprint gestures. The user interfaces inare used to illustrate the processes in. 41 41 FIGS.A-K 42 42 FIGS.A-C 41 41 FIGS.A-K 42 42 FIGS.A-C Below,illustrate exemplary user interfaces for displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control.are flow diagrams illustrating a method of displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control. The user interfaces inare used to illustrate the processes in. 44 44 FIGS.A-EE 45 45 FIGS.A-D 44 44 FIGS.A-EE 45 45 FIGS.A-D Below,illustrate exemplary user interfaces for configuring a second electronic device as an auxiliary display of a first electronic device.are flow diagrams illustrating a method of configuring a second electronic device as an auxiliary display of a first electronic device. The user interfaces inare used to illustrate the processes in. 47 47 FIGS.A-L 48 48 FIGS.A-B 47 47 FIGS.A-L 48 48 FIGS.A-B Below,illustrate exemplary user interfaces for changing beamforming parameters based on fingerprint orientation.are flow diagrams illustrating a method of changing beamforming parameters based on fingerprint orientation. The user interfaces inare used to illustrate the processes in. A number of different approaches to providing an intuitive user interface where inputs from one or more fingerprint sensors are used to manipulate a user interface of an electronic device are described below. Using one or more of these approaches (optionally in conjunction with each other) helps to provide a user interface that intuitively provides users with additional information and functionality, thereby reducing the user's cognitive burden and improving the human-machine interface. Such improvements in the human-machine interface enable users to use the device faster and more efficiently. For battery-operated devices, these improvements conserve power and increase the time between battery charges. For case of explanation, systems, methods and user interfaces for including illustrative examples of some of these approaches are described below, as follows:

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “includes,” “including,” “comprises,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [a stated condition or event] is detected” is, optionally, construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communications device, such as a mobile telephone, that also contains other functions, such as PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and/or touch pads), are, optionally, used. It should also be understood that, in some embodiments, the device is not a portable communications device, but is a desktop computer with a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).

In the discussion that follows, an electronic device that includes a display and a touch-sensitive surface is described. It should be understood, however, that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse and/or a joystick.

The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk authoring application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an e-mail application, an instant messaging application, a workout support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.

The various applications that are executed on the device optionally use at least one common physical user-interface device, such as the touch-sensitive surface. One or more functions of the touch-sensitive surface as well as corresponding information displayed on the device are, optionally, adjusted and/or varied from one application to the next and/or within a respective application. In this way, a common physical architecture (such as the touch-sensitive surface) of the device optionally supports the variety of applications with user interfaces that are intuitive and transparent to the user.

1 FIG.A 100 112 112 100 102 122 120 118 108 110 111 113 106 116 124 100 164 100 165 100 112 100 100 167 100 112 100 355 300 103 Attention is now directed toward embodiments of portable devices with touch-sensitive displays.is a block diagram illustrating portable multifunction devicewith touch-sensitive displaysin accordance with some embodiments. Touch-sensitive displayis sometimes called a “touch screen” for convenience, and is sometimes known as or called a touch-sensitive display system. Deviceincludes memory(which optionally includes one or more computer readable storage mediums), memory controller, one or more processing units (CPU's), peripherals interface, RF circuitry, audio circuitry, speaker, microphone, input/output (I/O) subsystem, other input or control devices, and external port. Deviceoptionally includes one or more optical sensors. Deviceoptionally includes one or more intensity sensorsfor detecting intensity of contacts on device(e.g., a touch-sensitive surface such as touch-sensitive display systemof device). Deviceoptionally includes one or more tactile output generatorsfor generating tactile outputs on device(e.g., generating tactile outputs on a touch-sensitive surface such as touch-sensitive display systemof deviceor touchpadof device). These components optionally communicate over one or more communication buses or signal lines.

As used in the specification and claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch sensitive surface, or to a substitute (proxy) for the force or pressure of a contact on the touch sensitive surface. The intensity of a contact has a range of values that includes at least four distinct values and more typically includes hundreds of distinct values (e.g., at least 256). Intensity of a contact is, optionally, determined (or measured) using various approaches and various sensors or combinations of sensors. For example, one or more force sensors underneath or adjacent to the touch-sensitive surface are, optionally, used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., a weighted average) to determine an estimated force of a contact. Similarly, a pressure-sensitive tip of a stylus is, optionally, used to determine a pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes thereto, the capacitance of the touch-sensitive surface proximate to the contact and/or changes thereto, and/or the resistance of the touch-sensitive surface proximate to the contact and/or changes thereto are, optionally, used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the substitute measurements for contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurements). In some implementations, the substitute measurements for contact force or pressure are converted to an estimated force or pressure and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure).

As used in the specification and claims, the term “tactile output” refers to physical displacement of a device relative to a previous position of the device, physical displacement of a component (e.g., a touch-sensitive surface) of a device relative to another component (e.g., housing) of the device, or displacement of the component relative to a center of mass of the device that will be detected by a user with the user's sense of touch. For example, in situations where the device or the component of the device is in contact with a surface of a user that is sensitive to touch (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in physical characteristics of the device or the component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is, optionally, interpreted by the user as a “down click” or “up click” of a physical actuator button. In some cases, a user will feel a tactile sensation such as an “down click” or “up click” even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movements. As another example, movement of the touch-sensitive surface is, optionally, interpreted or sensed by the user as “roughness” of the touch-sensitive surface, even when there is no change in smoothness of the touch-sensitive surface. While such interpretations of touch by a user will be subject to the individualized sensory perceptions of the user, there are many sensory perceptions of touch that are common to a large majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an “up click,” a “down click,” “roughness”), unless otherwise stated, the generated tactile output corresponds to physical displacement of the device or a component thereof that will generate the described sensory perception for a typical (or average) user.

100 100 1 FIG.A It should be appreciated that deviceis only one example of a portable multifunction device, and that deviceoptionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown inare implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and/or application specific integrated circuits.

102 102 100 120 118 122 Memoryoptionally includes high-speed random access memory and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memoryby other components of device, such as CPUand the peripherals interface, is, optionally, controlled by memory controller.

118 120 102 120 102 100 Peripherals interfacecan be used to couple input and output peripherals of the device to CPUand memory. The one or more processorsrun or execute various software programs and/or sets of instructions stored in memoryto perform various functions for deviceand to process data.

118 120 122 104 In some embodiments, peripherals interface, CPU, and memory controllerare, optionally, implemented on a single chip, such as chip. In some other embodiments, they are, optionally, implemented on separate chips.

108 108 108 108 RF (radio frequency) circuitryreceives and sends RF signals, also called electromagnetic signals. RF circuitryconverts electrical signals to/from electromagnetic signals and communicates with communications networks and other communications devices via the electromagnetic signals. RF circuitryoptionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth. RF circuitryoptionally communicates with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication. The wireless communication optionally uses any of a plurality of communications standards, protocols and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VOIP), Wi-MAX, a protocol for e-mail (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

110 111 113 100 110 118 111 111 110 113 110 118 102 108 118 110 212 110 2 FIG. Audio circuitry, speaker, and microphoneprovide an audio interface between a user and device. Audio circuitryreceives audio data from peripherals interface, converts the audio data to an electrical signal, and transmits the electrical signal to speaker. Speakerconverts the electrical signal to human-audible sound waves. Audio circuitryalso receives electrical signals converted by microphonefrom sound waves. Audio circuitryconverts the electrical signal to audio data and transmits the audio data to peripherals interfacefor processing. Audio data is, optionally, retrieved from and/or transmitted to memoryand/or RF circuitryby peripherals interface. In some embodiments, audio circuitryalso includes a headset jack (e.g.,,). The headset jack provides an interface between audio circuitryand removable audio input/output peripherals, such as output-only headphones or a headset with both output (e.g., a headphone for one or both cars) and input (e.g., a microphone).

106 100 112 116 118 106 156 158 159 161 160 160 116 116 160 208 111 113 206 2 FIG. 2 FIG. I/O subsystemcouples input/output peripherals on device, such as touch screenand other input control devices, to peripherals interface. I/O subsystemoptionally includes display controller, optical sensor controller, intensity sensor controller, haptic feedback controllerand one or more input controllersfor other input or control devices. The one or more input controllersreceive/send electrical signals from/to other input or control devices. The other input control devicesoptionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and so forth. In some alternate embodiments, input controller(s)are, optionally, coupled to any (or none) of the following: a keyboard, infrared port, USB port, and a pointer device such as a mouse. The one or more buttons (e.g.,,) optionally include an up/down button for volume control of speakerand/or microphone. The one or more buttons optionally include a push button (e.g.,,).

112 156 112 112 Touch-sensitive displayprovides an input interface and an output interface between the device and a user. Display controllerreceives and/or sends electrical signals from/to touch screen. Touch screendisplays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively termed “graphics”). In some embodiments, some or all of the visual output corresponds to user-interface objects.

112 112 156 102 112 112 112 Touch screenhas a touch-sensitive surface, sensor or set of sensors that accepts input from the user based on haptic and/or tactile contact. Touch screenand display controller(along with any associated modules and/or sets of instructions in memory) detect contact (and any movement or breaking of the contact) on touch screenand converts the detected contact into interaction with user-interface objects (e.g., one or more soft keys, icons, web pages or images) that are displayed on touch screen. In an exemplary embodiment, a point of contact between touch screenand the user corresponds to a finger of the user.

112 112 156 112 Touch screenoptionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. Touch screenand display controlleroptionally detect contact and any movement or breaking thereof using any of a plurality of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch screen. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in the iPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, California.

112 112 Touch screenoptionally has a video resolution in excess of 100 dpi. In some embodiments, the touch screen has a video resolution of approximately 160 dpi. The user optionally makes contact with touch screenusing any suitable object or appendage, such as a stylus, a finger, and so forth. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures, which can be less precise than stylus-based input due to the larger area of contact of a finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer/cursor position or command for performing the actions desired by the user.

100 112 In some embodiments, in addition to the touch screen, deviceoptionally includes a touchpad (not shown) for activating or deactivating particular functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad is, optionally, a touch-sensitive surface that is separate from touch screenor an extension of the touch-sensitive surface formed by the touch screen.

100 162 162 Devicealso includes power systemfor powering the various components. Power systemoptionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.

100 164 158 106 164 164 143 164 100 112 1 FIG.A Deviceoptionally also includes one or more optical sensors.shows an optical sensor coupled to optical sensor controllerin I/O subsystem. Optical sensoroptionally includes charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. Optical sensorreceives light from the environment, projected through one or more lens, and converts the light to data representing an image. In conjunction with imaging module(also called a camera module), optical sensoroptionally captures still images or video. In some embodiments, an optical sensor is located on the back of device, opposite touch screen displayon the front of the device, so that the touch screen display is enabled for use as a viewfinder for still and/or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user's image is, optionally, obtained for videoconferencing while the user views the other video conference participants on the touch screen display.

100 165 159 106 165 165 112 100 112 100 1 FIG.A Deviceoptionally also includes one or more contact intensity sensors.shows a contact intensity sensor coupled to intensity sensor controllerin I/O subsystem. Contact intensity sensoroptionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensorreceives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is collocated with, or proximate to, a touch-sensitive surface (e.g., touch-sensitive display system). In some embodiments, at least one contact intensity sensor is located on the back of device, opposite touch screen displaywhich is located on the front of device.

100 166 166 118 166 160 106 112 1 FIG.A Deviceoptionally also includes one or more proximity sensors.shows proximity sensorcoupled to peripherals interface. Alternately, proximity sensoris coupled to input controllerin I/O subsystem. In some embodiments, the proximity sensor turns off and disables touch screenwhen the multifunction device is placed near the user's car (e.g., when the user is making a phone call).

100 167 161 106 167 165 133 100 100 112 100 100 100 112 100 1 FIG.A Deviceoptionally also includes one or more tactile output generators.shows a tactile output generator coupled to haptic feedback controllerin I/O subsystem. Tactile output generatoroptionally includes one or more electroacoustic devices such as speakers or other audio components and/or electromechanical devices that convert energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts electrical signals into tactile outputs on the device). Contact intensity sensorreceives tactile feedback generation instructions from haptic feedback moduleand generates tactile outputs on devicethat are capable of being sensed by a user of device. In some embodiments, at least one tactile output generator is collocated with, or proximate to, a touch-sensitive surface (e.g., touch-sensitive display system) and, optionally, generates a tactile output by moving the touch-sensitive surface vertically (e.g., in/out of a surface of device) or laterally (e.g., back and forth in the same plane as a surface of device). In some embodiments, at least one tactile output generator sensor is located on the back of device, opposite touch screen displaywhich is located on the front of device.

100 168 168 118 168 160 106 100 168 100 1 FIG.A Deviceoptionally also includes one or more accelerometers.shows accelerometercoupled to peripherals interface. Alternately, accelerometeris, optionally, coupled to an input controllerin I/O subsystem. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on an analysis of data received from the one or more accelerometers. Deviceoptionally includes, in addition to accelerometer(s), a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information concerning the location and orientation (e.g., portrait or landscape) of device.

100 169 169 118 169 160 106 169 169 112 112 169 100 169 1 FIG.A 1 FIG.A In some embodiments, devicealso includes (or is in communication with) one or more fingerprint sensors.shows fingerprint sensorcoupled to peripherals interface. Alternately, fingerprint sensorsare, optionally, coupled to an input controllerin I/O subsystem. However, in one common embodiment, fingerprint identification operations are performed using secured dedicated computing hardware (e.g., one or more processors, memory and/or communications busses) that has additional security features so as to enhance security of the fingerprint information determined by the fingerprint sensors. As used herein, a fingerprint sensor is a sensor that is capable of distinguishing fingerprint features (sometimes called “minutia features”) of the ridges and valleys of skin such as those found on the fingers and toes of humans. A fingerprint sensor can use any of a variety of techniques to distinguish the fingerprint features, including but not limited to: optical fingerprint imaging, ultrasonic fingerprint imaging, active capacitance fingerprint imaging and passive capacitance fingerprint imaging. In addition to distinguishing fingerprint features in fingerprints, in some embodiments, fingerprint sensoris capable of tracking movement of fingerprint features over time and thereby determining/characterizing movement of the fingerprint over time on the fingerprint sensor. While the fingerprint sensor (e.g., Fingerprint Sensor) inis shown as being separate from the touch-sensitive surface (e.g., Touch-Sensitive Display System), it should be understood that in some implementations, the touch-sensitive surface (e.g., Touch-Sensitive Display System) has a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges and is used as a fingerprint sensor instead of, or in addition to, a separate fingerprint sensor (e.g., Fingerprint Sensor). In some embodiments, deviceincludes a set of one or more orientation sensors that are used to determine an orientation of a finger or hand on or proximate to the device (e.g., an orientation of a finger that is over fingerprint sensor). Additionally, in some embodiments, the set of one or more orientation sensors are used in addition to or instead of a fingerprint sensor to detect rotation of a contact that is interacting with the device (e.g., in one or more of the methods described below, instead of using a fingerprint sensor to detect rotation of a fingerprint/contact, the set of one or more orientation sensors is used to detect rotation of the contact that includes the fingerprint, with or without detecting features of the fingerprint).

120 169 120 131 In some embodiments, features of fingerprints and comparisons between features of detected fingerprints and features of stored fingerprints are performed by secured dedicated computing hardware (e.g., one or more processors, memory and/or communications busses) that are separate from processors, so as to improve security of the fingerprint data generated, stored and processed by fingerprint sensor. In some embodiments, features of fingerprints and comparisons between features of detected fingerprints and features of stored fingerprints are performed by processorsusing fingerprint analysis module.

102 126 128 130 131 132 134 135 136 102 157 157 112 116 1 3 FIGS.A and In some embodiments, the software components stored in memoryinclude operating system, communication module (or set of instructions), contact/motion module (or set of instructions), fingerprint analysis module, graphics module (or set of instructions), text input module (or set of instructions), Global Positioning System (GPS) module (or set of instructions), and applications (or sets of instructions). Furthermore, in some embodiments memorystores device/global internal state, as shown in. Device/global internal stateincludes one or more of: active application state, indicating which applications, if any, are currently active; display state, indicating what applications, views or other information occupy various regions of touch screen display; sensor state, including information obtained from the device's various sensors and input control devices; and location information concerning the device's location and/or attitude.

126 Operating system(e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

128 124 108 124 124 Communication modulefacilitates communication with other devices over one or more external portsand also includes various software components for handling data received by RF circuitryand/or external port. External port(e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, or similar to and/or compatible with the 30-pin connector used on iPod (trademark of Apple Inc.) devices.

130 112 156 130 130 130 156 Contact/motion moduleoptionally detects contact with touch screen(in conjunction with display controller) and other touch sensitive devices (e.g., a touchpad or physical click wheel). Contact/motion moduleincludes various software components for performing various operations related to detection of contact, such as determining if contact has occurred (e.g., detecting a finger-down event), determining an intensity of the contact (e.g., the force or pressure of the contact or a substitute for the force or pressure of the contact), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining if the contact has ceased (e.g., detecting a finger-up event or a break in contact). Contact/motion modulereceives contact data from the touch-sensitive surface. Determining movement of the point of contact, which is represented by a series of contact data, optionally includes determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (a change in magnitude and/or direction) of the point of contact. These operations are, optionally, applied to single contacts (e.g., one finger contacts) or to multiple simultaneous contacts (e.g., “multitouch”/multiple finger contacts). In some embodiments, contact/motion moduleand display controllerdetect contact on a touchpad.

130 100 In some embodiments, contact/motion moduleuses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has “clicked” on an icon). In some embodiments at least a subset of the intensity thresholds are determined in accordance with software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of particular physical actuators and can be adjusted without changing the physical hardware of device). For example, a mouse “click” threshold of a trackpad or touch screen display can be set to any of a large range of predefined thresholds values without changing the trackpad or touch screen display hardware. Additionally, in some implementations a user of the device is provided with software settings for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting a plurality of intensity thresholds at once with a system-level click “intensity” parameter).

130 Contact/motion moduleoptionally detects a gesture input by a user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and/or intensities of detected contacts). Thus, a gesture is, optionally, detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger-down event followed by detecting a finger-up (lift off) event at the same position (or substantially the same position) as the finger-down event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event followed by detecting one or more finger-dragging events, and subsequently followed by detecting a finger-up (lift off) event.

132 112 Graphics moduleincludes various known software components for rendering and displaying graphics on touch screenor other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast or other visual property) of graphics that are displayed. As used herein, the term “graphics” includes any object that can be displayed to a user, including without limitation text, web pages, icons (such as user-interface objects including soft keys), digital images, videos, animations and the like.

132 132 156 In some embodiments, graphics modulestores data representing graphics to be used. Each graphic is, optionally, assigned a corresponding code. Graphics modulereceives, from applications etc., one or more codes specifying graphics to be displayed along with, if necessary, coordinate data and other graphic property data, and then generates screen image data to output to display controller.

133 167 100 100 Haptic feedback moduleincludes various software components for generating instructions used by tactile output generator(s)to produce tactile outputs at one or more locations on devicein response to user interactions with device.

134 132 137 140 141 147 Text input module, which is, optionally, a component of graphics module, provides soft keyboards for entering text in various applications (e.g., contacts module, e-mail client module, IM module, browser module, and any other application that needs text input).

135 138 143 GPS moduledetermines the location of the device and provides this information for use in various applications (e.g., to telephone modulefor use in location-based dialing, to camera moduleas picture/video metadata, and to applications that provide location-based services such as weather widgets, local yellow page widgets, and map/navigation widgets).

136 137 contacts module(sometimes called an address book or contact list); 138 telephone module; 139 video conferencing module; 140 e-mail client module; 141 instant messaging (IM) module; 142 workout support module; 143 camera modulefor still and/or video images; 144 image management module; 147 browser module; 148 calendar module; 149 149 1 149 2 149 3 149 4 149 5 149 6 widget modules, which optionally include one or more of: weather widget-, stocks widget-, calculator widget-, alarm clock widget-, dictionary widget-, and other widgets obtained by the user, as well as user-created widgets-; 150 149 6 widget creator modulefor making user-created widgets-; 151 search module; 152 video and music player module, which is, optionally, made up of a video player module and a music player module; 153 notes module; 154 map module; and/or 155 online video module. Applicationsoptionally include the following modules (or sets of instructions), or a subset or superset thereof:

136 102 Examples of other applicationsthat are, optionally, stored in memoryinclude other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

112 156 130 132 134 137 192 137 102 370 138 139 140 141 In conjunction with touch screen, display controller, contact module, graphics module, and text input module, contacts moduleare, optionally, used to manage an address book or contact list (e.g., stored in application internal stateof contacts modulein memoryor memory), including: adding name(s) to the address book; deleting name(s) from the address book; associating telephone number(s), e-mail address(es), physical address(es) or other information with a name; associating an image with a name; categorizing and sorting names; providing telephone numbers or e-mail addresses to initiate and/or facilitate communications by telephone, video conference, e-mail, or IM; and so forth.

108 110 111 113 112 156 130 132 134 138 137 In conjunction with RF circuitry, audio circuitry, speaker, microphone, touch screen, display controller, contact module, graphics module, and text input module, telephone moduleare, optionally, used to enter a sequence of characters corresponding to a telephone number, access one or more telephone numbers in address book, modify a telephone number that has been entered, dial a respective telephone number, conduct a conversation and disconnect or hang up when the conversation is completed. As noted above, the wireless communication optionally uses any of a plurality of communications standards, protocols and technologies.

108 110 111 113 112 156 164 158 130 132 134 137 138 139 In conjunction with RF circuitry, audio circuitry, speaker, microphone, touch screen, display controller, optical sensor, optical sensor controller, contact module, graphics module, text input module, contact list, and telephone module, videoconferencing moduleincludes executable instructions to initiate, conduct, and terminate a video conference between a user and one or more other participants in accordance with user instructions.

108 112 156 130 132 134 140 144 140 143 In conjunction with RF circuitry, touch screen, display controller, contact module, graphics module, and text input module, e-mail client moduleincludes executable instructions to create, send, receive, and manage e-mail in response to user instructions. In conjunction with image management module, e-mail client modulemakes it very easy to create and send e-mails with still or video images taken with camera module.

108 112 156 130 132 134 141 In conjunction with RF circuitry, touch screen, display controller, contact module, graphics module, and text input module, the instant messaging moduleincludes executable instructions to enter a sequence of characters corresponding to an instant message, to modify previously entered characters, to transmit a respective instant message (for example, using a Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for telephony-based instant messages or using XMPP, SIMPLE, or IMPS for Internet-based instant messages), to receive instant messages and to view received instant messages. In some embodiments, transmitted and/or received instant messages optionally include graphics, photos, audio files, video files and/or other attachments as are supported in a MMS and/or an Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both telephony-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

108 112 156 130 132 134 135 154 146 142 In conjunction with RF circuitry, touch screen, display controller, contact module, graphics module, text input module, GPS module, map module, and music player module, workout support moduleincludes executable instructions to create workouts (e.g., with time, distance, and/or calorie burning goals); communicate with workout sensors (sports devices); receive workout sensor data; calibrate sensors used to monitor a workout; select and play music for a workout; and display, store and transmit workout data.

112 156 164 158 130 132 144 143 102 102 In conjunction with touch screen, display controller, optical sensor(s), optical sensor controller, contact module, graphics module, and image management module, camera moduleincludes executable instructions to capture still images or video (including a video stream) and store them into memory, modify characteristics of a still image or video, or delete a still image or video from memory.

112 156 130 132 134 143 144 In conjunction with touch screen, display controller, contact module, graphics module, text input module, and camera module, image management moduleincludes executable instructions to arrange, modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and/or video images.

108 112 156 130 132 134 147 In conjunction with RF circuitry, touch screen, display system controller, contact module, graphics module, and text input module, browser moduleincludes executable instructions to browse the Internet in accordance with user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

108 112 156 130 132 134 140 147 148 In conjunction with RF circuitry, touch screen, display system controller, contact module, graphics module, text input module, e-mail client module, and browser module, calendar moduleincludes executable instructions to create, display, modify, and store calendars and data associated with calendars (e.g., calendar entries, to do lists, etc.) in accordance with user instructions.

108 112 156 130 132 134 147 149 149 1 149 2 149 3 149 4 149 5 149 6 In conjunction with RF circuitry, touch screen, display system controller, contact module, graphics module, text input module, and browser module, widget modulesare mini-applications that are, optionally, downloaded and used by a user (e.g., weather widget-, stocks widget-, calculator widget-, alarm clock widget-, and dictionary widget-) or created by the user (e.g., user-created widget-). In some embodiments, a widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, a widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widgets).

108 112 156 130 132 134 147 150 In conjunction with RF circuitry, touch screen, display system controller, contact module, graphics module, text input module, and browser module, the widget creator moduleare, optionally, used by a user to create widgets (e.g., turning a user-specified portion of a web page into a widget).

112 156 130 132 134 151 102 In conjunction with touch screen, display system controller, contact module, graphics module, and text input module, search moduleincludes executable instructions to search for text, music, sound, image, video, and/or other files in memorythat match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

112 156 130 132 110 111 108 147 152 112 124 100 In conjunction with touch screen, display system controller, contact module, graphics module, audio circuitry, speaker, RF circuitry, and browser module, video and music player moduleincludes executable instructions that allow the user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions to display, present or otherwise play back videos (e.g., on touch screenor on an external, connected display via external port). In some embodiments, deviceoptionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).

112 156 130 132 134 153 In conjunction with touch screen, display controller, contact module, graphics module, and text input module, notes moduleincludes executable instructions to create and manage notes, to do lists, and the like in accordance with user instructions.

108 112 156 130 132 134 135 147 154 In conjunction with RF circuitry, touch screen, display system controller, contact module, graphics module, text input module, GPS module, and browser module, map moduleare, optionally, used to receive, display, modify, and store maps and data associated with maps (e.g., driving directions; data on stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

112 156 130 132 110 111 108 134 140 147 155 124 141 140 In conjunction with touch screen, display system controller, contact module, graphics module, audio circuitry, speaker, RF circuitry, text input module, e-mail client module, and browser module, online video moduleincludes instructions that allow the user to access, browse, receive (e.g., by streaming and/or download), play back (e.g., on the touch screen or on an external, connected display via external port), send an e-mail with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module, rather than e-mail client module, is used to send a link to a particular online video.

102 102 Each of the above identified modules and applications correspond to a set of executable instructions for performing one or more functions described above and the methods described in this application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memoryoptionally stores a subset of the modules and data structures identified above. Furthermore, memoryoptionally stores additional modules and data structures not described above.

100 100 100 In some embodiments, deviceis a device where operation of a predefined set of functions on the device is performed exclusively through a touch screen and/or a touchpad. By using a touch screen and/or a touchpad as the primary input control device for operation of device, the number of physical input control devices (such as push buttons, dials, and the like) on deviceis, optionally, reduced.

100 100 The predefined set of functions that are performed exclusively through a touch screen and/or a touchpad optionally include navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates deviceto a main, home, or root menu from any user interface that is displayed on device. In such embodiments, a “menu button” is implemented using a touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

1 FIG.B 1 FIG.A 3 FIG. 102 370 170 126 136 1 137 13 155 380 390 is a block diagram illustrating exemplary components for event handling in accordance with some embodiments. In some embodiments, memory(in) or() includes event sorter(e.g., in operating system) and a respective application-(e.g., any of the aforementioned applications-,,-).

170 136 1 191 136 1 170 171 174 136 1 192 112 157 170 192 170 191 Event sorterreceives event information and determines the application-and application viewof application-to which to deliver the event information. Event sorterincludes event monitorand event dispatcher module. In some embodiments, application-includes application internal state, which indicates the current application view(s) displayed on touch sensitive displaywhen the application is active or executing. In some embodiments, device/global internal stateis used by event sorterto determine which application(s) is (arc) currently active, and application internal stateis used by event sorterto determine application viewsto which to deliver event information.

192 136 1 136 1 136 1 In some embodiments, application internal stateincludes additional information, such as one or more of: resume information to be used when application-resumes execution, user interface state information that indicates information being displayed or that is ready for display by application-, a state queue for enabling the user to go back to a prior state or view of application-, and a redo/undo queue of previous actions taken by the user.

171 118 112 118 106 166 168 113 110 118 106 112 Event monitorreceives event information from peripherals interface. Event information includes information about a sub-event (e.g., a user touch on touch-sensitive display, as part of a multi-touch gesture). Peripherals interfacetransmits information it receives from I/O subsystemor a sensor, such as proximity sensor, accelerometer(s), and/or microphone(through audio circuitry). Information that peripherals interfacereceives from I/O subsystemincludes information from touch-sensitive displayor a touch-sensitive surface.

171 118 118 118 In some embodiments, event monitorsends requests to the peripherals interfaceat predetermined intervals. In response, peripherals interfacetransmits event information. In other embodiments, peripheral interfacetransmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and/or for more than a predetermined duration).

170 172 173 In some embodiments, event sorteralso includes a hit view determination moduleand/or an active event recognizer determination module.

172 112 Hit view determination moduleprovides software procedures for determining where a sub-event has taken place within one or more views, when touch sensitive displaydisplays more than one view. Views are made up of controls and other elements that a user can see on the display.

Another aspect of the user interface associated with an application is a set of views, sometimes herein called application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of a respective application) in which a touch is detected optionally correspond to programmatic levels within a programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is, optionally, called the hit view, and the set of events that are recognized as proper inputs are, optionally, determined based, at least in part, on the hit view of the initial touch that begins a touch-based gesture.

172 172 Hit view determination modulereceives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, hit view determination moduleidentifies a hit view as the lowest view in the hierarchy which should handle the sub-event. In most circumstances, the hit view is the lowest level view in which an initiating sub-event occurs (i.e., the first sub-event in the sequence of sub-events that form an event or potential event). Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

173 173 173 Active event recognizer determination moduledetermines which view or views within a view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination moduledetermines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination moduledetermines that all views that include the physical location of a sub-event are actively involved views, and therefore determines that all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if touch sub-events were entirely confined to the area associated with one particular view, views higher in the hierarchy would still remain as actively involved views.

174 180 173 174 173 174 182 Event dispatcher moduledispatches the event information to an event recognizer (e.g., event recognizer). In embodiments including active event recognizer determination module, event dispatcher moduledelivers the event information to an event recognizer determined by active event recognizer determination module. In some embodiments, event dispatcher modulestores in an event queue the event information, which is retrieved by a respective event receiver module.

126 170 136 1 170 170 102 130 In some embodiments, operating systemincludes event sorter. Alternatively, application-includes event sorter. In yet other embodiments, event sorteris a stand-alone module, or a part of another module stored in memory, such as contact/motion module.

136 1 190 191 191 136 1 180 191 180 180 136 1 190 176 177 178 179 170 190 176 177 178 192 191 190 176 177 178 191 In some embodiments, application-includes a plurality of event handlersand one or more application views, each of which includes instructions for handling touch events that occur within a respective view of the application's user interface. Each application viewof the application-includes one or more event recognizers. Typically, a respective application viewincludes a plurality of event recognizers. In other embodiments, one or more of event recognizersare part of a separate module, such as a user interface kit (not shown) or a higher level object from which application-inherits methods and other properties. In some embodiments, a respective event handlerincludes one or more of: data updater, object updater, GUI updater, and/or event datareceived from event sorter. Event handleroptionally utilizes or calls data updater, object updateror GUI updaterto update the application internal state. Alternatively, one or more of the application viewsincludes one or more respective event handlers. Also, in some embodiments, one or more of data updater, object updater, and GUI updaterare included in a respective application view.

180 179 170 180 182 184 180 183 188 A respective event recognizerreceives event information (e.g., event data) from event sorter, and identifies an event from the event information. Event recognizerincludes event receiverand event comparator. In some embodiments, event recognizeralso includes at least a subset of: metadata, and event delivery instructions(which optionally include sub-event delivery instructions).

182 170 Event receiverreceives event information from event sorter. The event information includes information about a sub-event, for example, a touch or a touch movement. Depending on the sub-event, the event information also includes additional information, such as location of the sub-event. When the sub-event concerns motion of a touch, the event information optionally also includes speed and direction of the sub-event. In some embodiments, events include rotation of the device from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation (also called device attitude) of the device.

184 184 186 186 187 1 187 2 187 187 1 187 2 112 190 Event comparatorcompares the event information to predefined event or sub-event definitions and, based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, event comparatorincludes event definitions. Event definitionscontain definitions of events (e.g., predefined sequences of sub-events), for example, event 1 (-), event 2 (-), and others. In some embodiments, sub-events in an eventinclude, for example, touch begin, touch end, touch movement, touch cancellation, and multiple touching. In one example, the definition for event 1 (-) is a double tap on a displayed object. The double tap, for example, comprises a first touch (touch begin) on the displayed object for a predetermined phase, a first lift-off (touch end) for a predetermined phase, a second touch (touch begin) on the displayed object for a predetermined phase, and a second lift-off (touch end) for a predetermined phase. In another example, the definition for event 2 (-) is a dragging on a displayed object. The dragging, for example, comprises a touch (or contact) on the displayed object for a predetermined phase, a movement of the touch across touch-sensitive display, and lift-off of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers.

187 184 112 112 184 190 190 184 In some embodiments, event definitionincludes a definition of an event for a respective user-interface object. In some embodiments, event comparatorperforms a hit test to determine which user-interface object is associated with a sub-event. For example, in an application view in which three user-interface objects are displayed on touch-sensitive display, when a touch is detected on touch-sensitive display, event comparatorperforms a hit test to determine which of the three user-interface objects is associated with the touch (sub-event). If each displayed object is associated with a respective event handler, the event comparator uses the result of the hit test to determine which event handlershould be activated. For example, event comparatorselects an event handler associated with the sub-event and the object triggering the hit test.

187 In some embodiments, the definition for a respective eventalso includes delayed actions that delay delivery of the event information until after it has been determined whether the sequence of sub-events does or does not correspond to the event recognizer's event type.

180 186 180 When a respective event recognizerdetermines that the series of sub-events do not match any of the events in event definitions, the respective event recognizerenters an event impossible, event failed, or event ended state, after which it disregards subsequent sub-events of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process sub-events of an ongoing touch-based gesture.

180 183 183 183 In some embodiments, a respective event recognizerincludes metadatawith configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery to actively involved event recognizers. In some embodiments, metadataincludes configurable properties, flags, and/or lists that indicate how event recognizers interact, or are enabled to interact, with one another. In some embodiments, metadataincludes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to varying levels in the view or programmatic hierarchy.

180 190 180 190 190 180 190 In some embodiments, a respective event recognizeractivates event handlerassociated with an event when one or more particular sub-events of an event are recognized. In some embodiments, a respective event recognizerdelivers event information associated with the event to event handler. Activating an event handleris distinct from sending (and deferred sending) sub-events to a respective hit view. In some embodiments, event recognizerthrows a flag associated with the recognized event, and event handlerassociated with the flag catches the flag and performs a predefined process.

188 In some embodiments, event delivery instructionsinclude sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver event information to event handlers associated with the series of sub-events or to actively involved views. Event handlers associated with the series of sub-events or with actively involved views receive the event information and perform a predetermined process.

176 136 1 176 137 145 177 136 1 177 178 178 132 In some embodiments, data updatercreates and updates data used in application-. For example, data updaterupdates the telephone number used in contacts module, or stores a video file used in video player module. In some embodiments, object updatercreates and updates objects used in application-. For example, object updatercreates a new user-interface object or updates the position of a user-interface object. GUI updaterupdates the GUI. For example, GUI updaterprepares display information and sends it to graphics modulefor display on a touch-sensitive display.

190 176 177 178 176 177 178 136 1 191 In some embodiments, event handler(s)includes or has access to data updater, object updater, and GUI updater. In some embodiments, data updater, object updater, and GUI updaterare included in a single module of a respective application-or application view. In other embodiments, they are included in two or more software modules.

100 It shall be understood that the foregoing discussion regarding event handling of user touches on touch-sensitive displays also applies to other forms of user inputs to operate multifunction deviceswith input-devices, not all of which are initiated on touch screens. For example, mouse movement and mouse button presses, optionally coordinated with single or multiple keyboard presses or holds; contact movements such as taps, drags, scrolls, etc., on touch-pads; pen stylus inputs; movement of the device; oral instructions; detected eye movements; biometric inputs; and/or any combination thereof are optionally utilized as inputs corresponding to sub-events which define an event to be recognized.

2 FIG. 100 112 200 202 203 100 illustrates a portable multifunction devicehaving a touch screenin accordance with some embodiments. The touch screen optionally displays one or more graphics within user interface (UI). In this embodiment, as well as others described below, a user is enabled to select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers(not drawn to scale in the figure) or one or more styluses(not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, right to left, upward and/or downward) and/or a rolling of a finger (from right to left, left to right, upward and/or downward) that has made contact with device. In some implementations or circumstances, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

100 204 204 136 100 112 204 169 1 204 204 169 2 204 169 1 204 169 2 112 112 169 1 169 2 100 100 Deviceoptionally also includes one or more physical buttons, such as “home” or menu button. As described previously, menu buttonis, optionally, used to navigate to any applicationin a set of applications that are, optionally executed on device. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touch screen. In some embodiments buttonincludes an integrated fingerprint sensor-for identifying a fingerprint that is interacting with buttonand/or detecting movement of the fingerprint on button. Device also, optionally, includes one or more other fingerprint sensors-that are separate from buttonand are used instead of or in conjunction with a fingerprint sensor-integrated into buttonto identify a user interacting with the device and/or detect motion of the fingerprint. Additionally, one or more of the other fingerprint sensors-are optionally associated with a button (e.g., a pressure sensitive region that is activated by detecting an input with an intensity above an activation intensity threshold or a physical actuator that moves in response force applied by a user). In implementations where the touch-sensitive surface (e.g., Touch Screen) has a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, the touch-sensitive surface (e.g., Touch Screen) is optionally used as a fingerprint sensor instead of, or in addition to, a separate fingerprint sensor (e.g., Fingerprint Sensors-or-). In some embodiments, deviceincludes a set of one or more orientation sensors that are used to determine an orientation of a hand on device.

100 112 204 206 208 210 212 124 206 100 113 100 165 112 167 100 In one embodiment, deviceincludes touch screen, menu button, push buttonfor powering the device on/off and locking the device, volume adjustment button(s), Subscriber Identity Module (SIM) card slot, head set jack, and docking/charging external port. Push buttonis, optionally, used to turn the power on/off on the device by depressing the button and holding the button in the depressed state for a predefined time interval; to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed; and/or to unlock the device or initiate an unlock process. In an alternative embodiment, devicealso accepts verbal input for activation or deactivation of some functions through microphone. Devicealso, optionally, includes one or more contact intensity sensorsfor detecting intensity of contacts on touch screenand/or one or more tactile output generatorsfor generating tactile outputs for a user of device.

3 FIG. 1 FIG.A 1 FIG.A 1 FIG.A 300 300 300 310 360 370 320 320 300 330 340 330 350 355 357 300 167 359 165 169 355 355 359 300 169 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments. Deviceneed not be portable. In some embodiments, deviceis a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a child's learning toy), a gaming system, or a control device (e.g., a home or industrial controller). Devicetypically includes one or more processing units (CPU's), one or more network or other communications interfaces, memory, and one or more communication busesfor interconnecting these components. Communication busesoptionally include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. Deviceincludes input/output (I/O) interfacecomprising display, which is typically a touch screen display. I/O interfacealso optionally includes a keyboard and/or mouse (or other pointing device)and touchpad, tactile output generatorfor generating tactile outputs on device(e.g., similar to tactile output generator(s)described above with reference to), sensors(e.g., optical, acceleration, proximity, touch-sensitive, and/or contact intensity sensors similar to contact intensity sensor(s)described above with reference to, and/or fingerprint sensors similar to fingerprint sensor(s)described above with reference to). Additionally, in implementations where the touch-sensitive surface (e.g., Touchpad) has a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, the touch-sensitive surface (e.g., Touchpad) is optionally used as a fingerprint sensor instead of, or in addition to, a separate fingerprint sensor (e.g., one of sensors). In some embodiments, deviceincludes a set of one or more orientation sensors that are used to determine an orientation of a finger or hand on or proximate to the device (e.g., an orientation of a finger that is over fingerprint sensor). Additionally, in some embodiments, the set of one or more orientation sensors are used in addition to or instead of a fingerprint sensor to detect rotation of a contact that is interacting with the device. For example, in one or more of the methods described below, instead of using a fingerprint sensor to detect rotation of a fingerprint/contact, the set of one or more orientation sensors is used to detect rotation of the contact that includes the fingerprint, with or without detecting features of the fingerprint.

370 370 310 370 102 100 370 102 100 370 300 380 382 384 386 388 390 102 100 1 FIG.A 1 FIG.A Memoryincludes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memoryoptionally includes one or more storage devices remotely located from CPU(s). In some embodiments, memorystores programs, modules, and data structures analogous to the programs, modules, and data structures stored in memoryof portable multifunction device(), or a subset thereof. Furthermore, memoryoptionally stores additional programs, modules, and data structures not present in memoryof portable multifunction device. For example, memoryof deviceoptionally stores drawing module, presentation module, word processing module, website creation module, disk authoring module, and/or spreadsheet module, while memoryof portable multifunction device() optionally does not store these modules.

3 FIG. 370 370 Each of the above identified elements inare, optionally, stored in one or more of the previously mentioned memory devices. Each of the above identified modules corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memoryoptionally stores a subset of the modules and data structures identified above. Furthermore, memoryoptionally stores additional modules and data structures not described above.

100 Attention is now directed towards embodiments of user interfaces (“UI”) that is, optionally, implemented on portable multifunction device.

4 FIG.A 100 300 400 402 Signal strength indicator(s)for wireless communication(s), such as cellular and Wi-Fi signals; 404 Time; 405 Bluetooth indicator; 406 Battery status indicator; 408 416 138 414 Iconfor telephone module, labeled “Phone,” which optionally includes an indicatorof the number of missed calls or voicemail messages; 418 140 410 Iconfor e-mail client module, labeled “Mail,” which optionally includes an indicatorof the number of unread e-mails; 420 147 Iconfor browser module, labeled “Browser;” and 422 152 152 Iconfor video and music player module, also referred to as iPod (trademark of Apple Inc.) module, labeled “iPod;” and Traywith icons for frequently used applications, such as: 424 141 Iconfor IM module, labeled “Text;” 426 148 Iconfor calendar module, labeled “Calendar;” 428 144 Iconfor image management module, labeled “Photos;” 430 143 Iconfor camera module, labeled “Camera;” 432 155 Iconfor online video module, labeled “Online Video” 434 149 2 Iconfor stocks widget-, labeled “Stocks;” 436 154 Iconfor map module, labeled “Map;” 438 149 1 Iconfor weather widget-, labeled “Weather;” 440 149 4 Iconfor alarm clock widget-, labeled “Clock;” 442 142 Iconfor workout support module, labeled “Workout Support;” 444 153 Iconfor notes module, labeled “Notes;” and 446 100 136 Iconfor a settings application or module, which provides access to settings for deviceand its various applications. Icons for other applications, such as: illustrates an exemplary user interface for a menu of applications on portable multifunction devicein accordance with some embodiments. Similar user interfaces are, optionally, implemented on device. In some embodiments, user interfaceincludes the following elements, or a subset or superset thereof:

4 FIG.A 422 152 It should be noted that the icon labels illustrated inare merely exemplary. For example, iconfor video and music player moduleare labeled “Music” or “Music Player.” Other labels are, optionally, used for various application icons. In some embodiments, a label for a respective application icon includes a name of an application corresponding to the respective application icon. In some embodiments, a label for a particular application icon is distinct from a name of an application corresponding to the particular application icon.

4 FIG.B 3 FIG. 3 FIG. 1 FIG.A 300 451 355 450 112 359 1 359 2 359 169 451 451 359 1 359 2 300 359 3 451 357 300 illustrates an exemplary user interface on a device (e.g., device,) with a touch-sensitive surface(e.g., a tablet or touchpad,) that is separate from the display(e.g., touch screen display) with an integrated-(or separate-) fingerprint sensor (e.g., one or more of sensorsthat operates in an analogous manner to fingerprint sensorin). Additionally, in implementations where the touch-sensitive surfacehas a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, the touch-sensitive surfaceis optionally used as a fingerprint sensor instead of, or in addition to, a distinct fingerprint sensor (e.g., integrated fingerprint sensor-or separate fingerprint sensor-). Devicealso, optionally, includes one or more contact intensity sensors (e.g., one or more of sensors-) for detecting intensity of contacts on touch-sensitive surfaceand/or one or more tactile output generatorsfor generating tactile outputs for a user of device.

112 451 452 453 450 460 462 451 468 462 470 460 462 451 450 4 FIG.B 4 FIG.B 4 FIG.B 4 FIG.B 4 FIG.B 4 460 FIG.B, 4 FIG.B 4 FIG.B Although some of the examples which follow will be given with reference to inputs on touch screen display(where the touch sensitive surface and the display are combined), in some embodiments, the device detects inputs on a touch-sensitive surface that is separate from the display, as shown in. In some embodiments the touch sensitive surface (e.g.,in) has a primary axis (e.g.,in) that corresponds to a primary axis (e.g.,in) on the display (e.g.,). In accordance with these embodiments, the device detects contacts (e.g.,andin) with the touch-sensitive surfaceat locations that correspond to respective locations on the display (e.g., incorresponds toandcorresponds to). In this way, user inputs (e.g., contactsand, and movements thereof) detected by the device on the touch-sensitive surface (e.g.,in) are used by the device to manipulate the user interface on the display (e.g.,in) of the multifunction device when the touch-sensitive surface is separate from the display. It should be understood that similar methods are, optionally, used for other user interfaces described herein.

Additionally, while the following examples are given primarily with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures), it should be understood that, in some embodiments, one or more of the finger inputs are replaced with input from another input device (e.g., a mouse based input or stylus input). For example, a swipe gesture is, optionally, replaced with a mouse click (e.g., instead of a contact) followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is, optionally, replaced with a mouse click while the cursor is located over the location of the tap gesture (e.g., instead of detection of the contact followed by ceasing to detect the contact). Similarly, when multiple user inputs are simultaneously detected, it should be understood that multiple computer mice are, optionally, used simultaneously, or a mouse and finger contacts are, optionally, used simultaneously.

355 451 112 112 3 FIG. 4 FIG.B 1 FIG.A 4 FIG.A As used herein, the term “focus selector” refers to an input element that indicates a current part of a user interface with which a user is interacting. In some implementations that include a cursor or other location marker, the cursor acts as a “focus selector,” so that when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpadinor touch-sensitive surfacein) while the cursor is over a particular user interface element (e.g., a button, window, slider or other user interface element), the particular user interface element is adjusted in accordance with the detected input. In some implementations that include a touch-screen display (e.g., touch-sensitive display systeminor touch screenin) that enables direct interaction with user interface elements on the touch-screen display, a detected contact on the touch-screen acts as a “focus selector,” so that when an input (e.g., a press input by the contact) is detected on the touch-screen display at a location of a particular user interface element (e.g., a button, window, slider or other user interface element), the particular user interface element is adjusted in accordance with the detected input. In some implementations focus is moved from one region of a user interface to another region of the user interface without corresponding movement of a cursor or movement of a contact on a touch-screen display (e.g., by using a tab key or arrow keys to move focus from one button to another button); in these implementations, the focus selector moves in accordance with movement of focus between different regions of the user interface. Without regard to the specific form taken by the focus selector, the focus selector is generally the user interface element (or contact on a touch-screen display) that is controlled by the user so as to communicate the user's intended interaction with the user interface (e.g., by indicating, to the device, the element of the user interface with which the user is intending to interact). For example, the location of a focus selector (e.g., a cursor, a contact or a selection box) over a respective button while a press input is detected on the touch-sensitive surface (e.g., a touchpad or touch screen) will indicate that the user is intending to activate the respective button (as opposed to other user interface elements shown on a display of the device).

4 FIG.C shows a schematic representation of a two hands with associated fingerprints LP (“left pinky” fingerprint), LR (“left ring” fingerprint), LM (“left middle” fingerprint), LI (“left index” fingerprint), LT (“left thumb” fingerprint), RT (“right thumb” fingerprint), RI (“right index” fingerprint), RM (“right middle” fingerprint), RR (“right ring” fingerprint), RP (“right pinky” fingerprint). These abbreviations will be used with reference to other figures showing examples of interactions with a fingerprint sensor. For some of the methods described herein, one or more fingerprints of a user are registered by collecting information about the fingerprint that would enable the fingerprint to be identified. These registered fingerprints or pre-registered fingerprints are also sometimes referred to as enrolled fingerprints. In many situations described below, fingerprints detected on a fingerprint sensor are compared against previously registered fingerprints (e.g., enrolled fingerprints).

4 FIG.D 4 FIG.D 4 FIG.D 169 204 shows two different types of rotation of a fingerprint on a fingerprint sensor (e.g., fingerprint sensorthat is integrated into button). On the left side of, is an example of a fingerprint “revolving” around a fingerprint sensor, where a centroid of the fingerprint moves in a looping motion (e.g., a circular-type motion) around a center of the fingerprint sensor. On the right side of, is an example of a fingerprint “twisting” on a fingerprint sensor, where a primary axis of the fingerprint changes orientation relative to the fingerprint sensor. These terms (e.g., “revolving” and “twisting”) will be used to described different types of rotation of a fingerprint on a fingerprint sensor with reference to other figures showing examples of interactions with a fingerprint sensor.

4 FIG.D As shown in, in some embodiments, the fingerprint sensor is smaller than, or approximately the same size as, an average fingerprint. Thus, in some embodiments, the fingerprint sensor detects movement of the fingerprint (e.g., fingerprint gestures) by detecting movement of fingerprint features of the fingerprint instead of or in addition to detecting movement of edges of the fingerprint. In other words, in some implementations, the fingerprint sensor detects movement of the fingerprint not by determining movement of an outline of the fingerprint (e.g., a “contact”) but by detecting movement of ridges (or specific minutia points in the ridges) of the fingerprint over the fingerprint sensor. Without regard to whether the fingerprint sensor is larger or smaller than the fingerprint, detecting movement of a fingerprint (e.g., fingerprint gestures) based on movement of fingerprint features instead of, or in addition to, detecting movement of the fingerprint based on movement of an edge or outline of the fingerprint enables movement of the fingerprint to be tracked in much greater detail and provides the ability to track fingerprint movement even when a fingerprint is covering all or a large portion of the fingerprint sensor.

Many electronic devices have graphical user interfaces that allow users to perform a variety of functions and operations associated with the respective user interfaces (e.g., home screen or application launch interfaces with user-selectable icons for selecting applications to be launched; application interfaces associated with applications such as maps for performing navigation operations, calculators for performing computation, photo viewers for viewing digital photographs, search applications, camera applications and the like; system user interfaces associated with various system level functions). While interacting with such interfaces, the user frequently desires to navigate through a given user interface; or to switch from one user interface to another (e.g., from a calculator application interface to a photo viewing application interface; or from a home screen interface to a camera application interface; and the like) or to switch from an application user interface to a system user interface (e.g., to a multi-tasking user interface). Some approaches to navigating through a user interface involve making long sustained swiping gestures across a substantial portion of a touch-sensitive surface (such a touch screen display). Some approaches for switching between application user interfaces (e.g., switching between applications) involve returning to a home screen and then toggling through pages of the home screen to view an icon corresponding to a desired application, and then selecting the icon corresponding to the desired application to launch the desired application. Similarly, some approaches to switching to a multi-tasking user interface involve making multiple concurrent contacts with the home or menu button in order to display a system user interface. These approaches are cumbersome and require the user to either make sustained movements of one or more finger contacts with a touch-sensitive surface or to make multiple sequential contacts with the device, which can take a long time or be difficult for a user to remember.

The disclosed embodiments provide a convenient and efficient method of interacting with user interfaces (e.g., either navigating through a current user interface or displaying a different user interface) through the use of a single short movement of a fingerprint (e.g., detected on a fingerprint sensor located on the device) in one of two directions. While displaying a first user interface (such as an application user interface), in response to detecting a fingerprint movement on the fingerprint sensor in a first direction (e.g., horizontally, from right to left, or left to right), the device navigates through the first user interface (e.g., translates or scrolls at least a portion of the content in the first user interface) and in response to detecting a fingerprint movement on the fingerprint sensor in a second direction (e.g., a vertical direction, upward), the device displays a second user interface (e.g., a multi-tasking user interface or the user interface corresponding to a most recently used application). As a result, the user has the improved convenience of interacting with one or more user interfaces in two different ways—e.g., cither navigating through a first (or current) user interface or displaying a second user interface-simply by performing a gesture that involves the movement of a fingerprint in one of two directions on a fingerprint sensor.

450 451 100 112 167 5 5 6 6 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 450 451 359 2 359 1 359 1 450 112 169 359 1 359 2 112 112 5 1 FIG.A 4 FIG.B 4 FIG.B 5 5 6 6 FIGS.A-N andA-C 5 5 FIGS.A-N 5 5 FIGS.A-N 5 5 FIGS.A-N 5 5 FIGS.A-N In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-N andA-C includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to display, a separate touch-sensitive surfaceand a separate fingerprint sensor-, however analogous operations are, optionally, performed on a device with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display. Additionally, analogous operations are, optionally, performed on a device with a touch screenin response to detecting the contacts described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon the touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of cursorN.

5 FIG.A 5 FIG.A 5 FIG.A 502 100 502 100 169 169 502 169 204 502 illustrates a first user interface (e.g., an application user interface, such as a map application user interface) with content (e.g., a map and navigation instructions) that is displayed on a displayof a portable multi-function device. In some embodiments, displayis a touch-sensitive display with an array of touch sensors that are responsive to touch. As shown in, deviceincludes a fingerprint sensor. In some embodiments, fingerprint sensoris separate from display. Further, as shown in, in some embodiments, fingerprint sensoris integrated into a physical button (e.g., home button or menu button). In some embodiments, the device includes a touch-sensitive surface (e.g., a touch sensitive surface collocated with the display in the form of a touch sensitive display, such as touch sensitive display; and/or a touch sensitive surface separate from the display on the device), and the fingerprint sensor is separate from the touch-sensitive surface. In such embodiments, while the fingerprint sensor is responsive to the location, proximity and/or touch of a finger contact (e.g., in the form of a fingerprint), and is optionally responsive to the movement of the finger contact (e.g., in the form of a movement of the fingerprint), the spatial resolution of the fingerprint sensor (e.g., to detect the spatial location and/or movement of the finger) is greater than (e.g., more sensitive than) the resolution of conventional touch-sensitive surfaces; thereby making the fingerprint sensor more sensitive to small-magnitude spatial motion of the finger than conventional touch-sensitive surfaces.

5 5 FIGS.B-C 5 FIG.B 5 FIG.B 5 FIG.C 5 FIG.C 5 FIG.B 169 502 510 512 169 512 510 169 510 510 510 illustrate, while displaying a first user interface, detecting a movement of a fingerprint on fingerprint sensorin a first direction (e.g., in a lateral or horizontal direction or along a short-axis of the multi-purpose device; from right to left); and in response, navigating through the first user interface. In some embodiments, as shown in, while the first user interface (e.g., the map user interface) is displayed on display, a movement of a fingerprint (e.g., fingerprintand associated movement), is detected on fingerprint sensor. In response to detecting movementof fingerprinton fingerprint sensor(e.g., from a location of fingerprintshown into a location of fingerprintshown in), the device navigates through the first user interface For example, the device translates at least a portion of the first user interface in the direction of movement of fingerprint(e.g., from right to left) to display additional portions of the first user interface, as shown in, that were not previously displayed in. Thus, in some embodiments, while displaying a first user interface (e.g., corresponding to a first application or to an application launch user interface), in response to detecting a movement of a fingerprint in a first direction (e.g., in a lateral or horizontal direction or along the short-axis of the multi-purpose device; from right to left) on a fingerprint sensor, the device navigates through the first user interface (e.g., translates at least a portion of the content displayed on the first user interface).

5 5 FIGS.D-E 5 FIG.D 5 FIG.D 5 FIG.D 5 FIG.E 5 FIG.E 169 514 516 169 100 516 100 516 514 514 514 517 517 illustrate, while displaying a first user interface, detecting a movement of a fingerprint on fingerprint sensorin a second direction (e.g., in a vertical direction or along the long-axis of the multi-purpose device; upward); and in response, displaying a second user interface different from the first user interface. As shown in, in some embodiments, while displaying the first user interface (e.g., map application), the movement of a fingerprint (e.g., fingerprintand associated movement) is detected on fingerprint sensorof device, the corresponding movementbeing in a second direction (e.g., in a vertical direction or along the long-axis of the multi-purpose device; upward, with reference to the orientation of deviceshown in). In accordance with a determination that movementof fingerprintis in the second direction (e.g., in a vertical upward direction, from the location of fingerprintshown into the location of fingerprintshown in), a second user interface (e.g., multi-tasking user interfaceincluding user selectable icons corresponding to a set of most-recently used applications, such as a notes application, a camera application, a photo viewer application, and a voice memo recording application; where selection of one of the icons in the multi-tasking user interfacewill cause the device to run an application corresponding to the selected icon as a currently active application, sometimes called a foreground application) is displayed, as shown in. For example, selection of the “notes” icon will cause the device to run the notes application as a currently active application and selection of the “camera” icon will cause the device to run the camera application as a currently active application.

5 5 FIGS.D-E 5 FIG.E 5 FIG.E 517 517 517 Accordingly, in some embodiments, the application user interface (e.g., the map application user interface shown in) is at least partially replaced by the second user interface (e.g., the map application is partially displaced upward and/or at least partially concealed by multi-tasking user interfaceshown in). In some embodiments, upon displaying the second user interface (e.g., multi-tasking user interface,) one or more interactive elements in the application user interface are disabled from user-interaction (e.g., while multi-tasking user interfaceis displayed, the map application would be disabled from user-interaction and the user would not be able to scroll, navigate through, or otherwise interact with the map application). Thus, in some embodiments, while displaying a first user interface (e.g., corresponding to a first application), in response to detecting a movement of a fingerprint in a second direction (e.g., a vertical direction or along the long-axis of the multi-purpose device; upward), the device displays a second system user interface (e.g., a multi-tasking user interface).

5 5 FIGS.F-G 5 FIG.F 5 FIG.F 5 5 FIGS.F-G 5 FIG.G 5 FIG.F 5 FIG.F 5 FIG.G 5 FIG.F 169 517 518 520 169 100 100 520 518 517 520 518 illustrate, while displaying the second user interface, detecting a movement of a fingerprint on fingerprint sensorin a first direction (e.g., in a lateral or horizontal direction or along the short-axis of the multi-purpose device; from right to left); and in response, navigating through the second user interface. In some embodiments, while displaying the second user interface (e.g., multi-tasking user interface,), the movement of a fingerprint (e.g., fingerprintand associated movement) is detected on fingerprint sensorof device, the corresponding movement being in the first direction (e.g., in a lateral or horizontal direction or along the short-axis of the multi-purpose device; from right to left with reference to the orientation of deviceshown in). In response to detecting movementof fingerprint, the device navigates through the second user interface (e.g., multi-tasking user interface,). For example, the device translates at least a portion of the second user interface in the direction of movementof fingerprintto display additional portions of the second user interface (e.g., additional icons for a message application, calendar application, and clock application as shown in, that were not previously displayed in, and the icons for the notes application, the camera application, the photos application and the voice memo recording application that were previously displayed incease to be displayed in). In some embodiments, in response to detecting movement of the fingerprint from left to right on the fingerprint sensor, the navigation through the second user interface would be reversed, so that the icons for the notes application, the camera application, the photos application and the voice memo recording application that were previously displayed inwould be redisplayed.

5 5 FIGS.H-I 5 5 FIGS.H-I 5 FIG.H 5 5 FIGS.D-E 5 FIG.H 5 FIG.I 169 522 524 169 100 524 100 524 522 517 Further,illustrate, while displaying the second user interface, detecting a movement of a fingerprint on fingerprint sensorin a third direction (e.g., a vertical direction or along the long-axis of the multi-purpose device; downward; substantially opposite to the second direction); and in response, ceasing to display the second user interface. In some embodiments, as illustrated in, while displaying the second user interface, movement of a fingerprint (e.g., fingerprintand associated movement) is detected on fingerprint sensorof device, movementbeing in a third direction (e.g., in a vertical direction downward along the long-axis of the multi-purpose device with reference to the orientation of deviceshown in; a direction substantially opposite to the second direction described with reference to). In response to detecting movementof fingerprintin the third direction, the device ceases to display the second user interface (e.g., the multi-tasking user interfacepreviously shown in) and redisplays the first user interface or the map application interface (e.g., in its entirety), as shown in.

5 5 FIGS.J-K 5 5 FIGS.J-K 5 FIG.E 5 FIG.K 204 169 530 204 502 502 illustrate, while displaying the first user interface, detecting a double activation of the physical home button or menu button(e.g., into which fingerprint sensoris, optionally, integrated); and in response, displaying the second user interface. In some embodiments, as illustrated in, upon detecting a double activationof the physical home button or menu button(e.g., via two successive taps or press inputs occurring in quick succession with less than a predefined interval of time lapse between the two successive taps, for example with an interval of time less than 500 milliseconds, 300 milliseconds, 150 milliseconds, 50 milliseconds or some other reasonable time interval; with or without a finger lift-off between the two successive taps), the second interface (e.g., the multi-tasking user interface shown previously inwith user-selectable icons corresponding to recently used or active applications such as the notepad application, the camera application, the photo viewer application, and the voice memo recording application) is displayed on display, as shown inand the first user interface (e.g., the map application user interface) is at least partially concealed or slid off the displayand optionally deactivated from user-interaction.

5 5 FIGS.K-L 5 5 FIGS.K-L 5 FIG.K 5 FIG.L 204 169 532 204 204 502 illustrate, while displaying the second user interface, detecting an activation of the physical home button or menu button(e.g., into which fingerprint sensoris integrated); and in response, ceasing to display the second user interface (e.g., and redisplaying and reactivating the first user interface). In some embodiments, as illustrated in, upon detecting activationof the physical home button or menu button(e.g., via a tap or press input on the physical button), the second user interface (e.g., the multi-tasking user interface shown inwith user-selectable icons corresponding to recently used or active applications such as the notepad application, the camera application, the photo viewer application, and the voice memo recording application) is no longer displayed on display, as shown in. In some embodiments, the first user interface (e.g., the map application user interface) is redisplayed (e.g., in its entirety) and reactivated for user input (e.g., from its previously deactivated state).

5 5 FIGS.M-N 5 5 FIGS.M-N 5 FIG.M 5 FIG.N 5 FIG.N 204 169 204 534 204 204 502 204 illustrate, while displaying the first user interface, detecting an activation of the physical home button or menu button(e.g., into which fingerprint sensoris integrated); and in response, replacing the first user interface (e.g., ceasing to display the first user interface) with a predefined user interface (e.g., a home screen or application launch interface associated with the physical home button or menu button). In some embodiments, as illustrated in, upon detecting activationof the physical home button or menu button(e.g., via a tap or press input on the physical button), the first user interface (e.g., the multi-tasking user interface shown in) is no longer displayed on display, as shown in. In some embodiments, the first user interface (e.g., the map application user interface) is replaced (e.g., in its entirety) with a predefined user interface (e.g., a home screen or application launch user interface, as shown in). In some embodiments, activation of buttonwhile the home screen is displayed would cause the device to display a predefined search user interface.

6 6 FIGS.A-C 3 FIG. 1 FIG.A 600 600 300 100 600 are flow diagrams illustrating a methodof interacting with user interfaces in accordance with movement of a finger on a fingerprint sensor in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

600 As described below, the methodprovides an intuitive way to interact with user interfaces. The method reduces the cognitive burden on a user when interacting with user interfaces, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to interact with user interfaces faster and more efficiently conserves power and increases the time between battery charges.

602 604 512 510 516 514 5 FIG.A 5 FIG.B 5 FIG.D The device displays (), on the display, a first user interface (e.g., an application user interface, such as a map application or navigation application user interface as shown in). While displaying the first user interface on the display, the device detects () movement of a fingerprint on the fingerprint sensor (e.g., movementof fingerprint,; movementof fingerprint,).

606 169 502 608 5 FIG.A In some embodiments, the fingerprint sensor is () separate from (e.g., not co-incident with) the display (e.g., fingerprint sensoris separate from display,). The device includes () a touch-sensitive surface; and the fingerprint sensor is separate from (e.g., not co-incident with) the touch-sensitive surface.

610 512 510 510 5 FIG.B 5 5 FIGS.B-C 5 5 FIGS.B-C In response to detecting () movement of the fingerprint on the fingerprint sensor: in accordance with a determination that the movement of the fingerprint is in a first direction (e.g., a primary component of movement of the fingerprint is horizontally rightward, or horizontally leftward as described above with reference to movementof fingerprint,), the device navigates through the first user interface (e.g., as described above with reference to navigation through the map interface shown in). In some embodiments, the device navigates back to a previous state, or previous page, or previous webpage, or previous photo (e.g., in response to detecting a rightward movement of the fingerprint) or forward to a next state, or next page, or next webpage, or next photo (e.g., in response to detecting a leftward movement of the fingerprint). In some embodiments the device scrolls the first user interface horizontally in accordance with the direction of movement of the fingerprint on the fingerprint sensor (e.g., the device translates at least a portion of the content on the first user interface, such as the map application user interface, in accordance with a direction of movement of the fingerprint horizontally from right to left, as described above with reference to the movement of fingerprintin).

516 514 517 517 502 517 5 FIG.D 5 5 FIGS.D-E 5 FIG.E 5 FIG.E In accordance with a determination that the movement of the fingerprint is in a second direction (e.g., a primary component of movement of the fingerprint vertically downward or vertically upward as described above with reference to movementof fingerprint,) different from the first direction, the device displays a second user interface different from the first user interface on the display (e.g., as described above with reference to multitasking user interfacein). In some embodiments, the device displays a multitasking user interface (e.g., multi-tasking user interface,; or a most recently used application). In some embodiments, displaying the second user interface includes replacing the first user interface with the second user interface. In some embodiments, displaying the second user interface includes ceasing to display at least a portion of the first user interface (e.g., the first user interface is at least partially covered with the second user interface or the first user interface slides at least partially off of the display so as to display the second user interface; as shown in, the map application interface is partially displaced off displayin order to display multi-tasking user interface).

612 512 510 516 514 5 5 FIGS.B-C 5 5 FIGS.D-E In some embodiments, the first direction is () perpendicular (or substantially perpendicular) to the second direction. For example, as described above with reference to movementof fingerprintin, the first direction is a lateral or horizontal direction or along the short-axis of the multi-purpose device, from right to left; as described above with reference to movementof fingerprintin, the second direction is a vertical direction or along the long-axis of the multi-purpose device; upward (e.g., perpendicular or substantially perpendicular to the first direction).

614 517 157 192 5 5 FIGS.A-D 5 FIG.E 5 FIG.E 112 an active application, which is currently displayed on display(or a corresponding application view is currently displayed on the display); 112 120 a background application (or background process), which is not currently displayed on display, but one or more application processes (e.g., instructions) for the corresponding application are being processed by one or more processors(i.e., running); 102 a suspended application, which is not currently running, and the application is stored in a volatile memory (e.g., DRAM, SRAM, DDR RAM, or other volatile random access solid state memory device of memory); and 102 a hibernated application, which is not running, and the application is stored in a non-volatile memory (e.g., one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices of memory). In some embodiments, the first user interface is () a user interface of a first application (e.g., map application user interface,); and the second user interface is a multitasking user interface (e.g., multi-tasking user interface,) that includes representations (e.g., selectable icons) of a plurality of concurrently open applications (e.g., a “notes” icon corresponding to a notepad application, a “camera” icon corresponding to a camera application, a “photos” icon corresponding to a photo viewer application, and a “voice memos” icon corresponding to a voice memo recording application, as shown in). In some embodiments, in response to detecting selection of a respective representation of one of the concurrently open applications, the device displays an open application that corresponds to the respective representation. In some embodiments, the multitasking user interface includes representations (e.g., selectable icons) of one or more recently used applications (e.g., applications which were recently open on the device but for which no retained state information is currently stored). As used in the specification and claims, the term “open application” refers to a software application with retained state information (e.g., as part of device/global internal stateand/or application internal state). An open application is any one of the following types of applications:

As used herein, the term “closed application” refers to software applications without retained state information (e.g., state information for closed applications is not stored in a memory of the device). Accordingly, closing an application includes stopping and/or removing application processes for the application and removing state information for the application from the memory of the device. Generally, opening a second application while in a first application (e.g., switching from the first application to the second application) does not close the first application. When the second application is displayed and the first application ceases to be displayed, the first application, which was an active application when displayed, may become a background application, suspended application, or hibernated application, but the first application remains an open application while its state information is retained by the device.

616 618 620 5 5 FIGS.B-C In some embodiments, navigating through the first user interface includes () navigating through the first user interface at a rate determined in accordance with a displacement of the fingerprint on the fingerprint sensor. For example, the device scrolls through or translates at least a portion of content displayed in the first user interface at a rate of 5 millimeters/second for each 1 millimeter of movement by the fingerprint on the fingerprint sensor from a starting position of the fingerprint on the fingerprint sensor. In some embodiments, navigating through the first user interface includes () navigating through the first user interface by an amount determined in accordance with a displacement of the fingerprint on the fingerprint sensor. For example, the device scrolls through or translates at least a portion of the content displayed in the first user interface by 5 millimeters for each 1 millimeter of movement by the fingerprint on the fingerprint sensor. In some embodiments, navigating through the first user interface includes () navigating through the first user interface in a direction determined in accordance with a direction of movement of the fingerprint on the fingerprint sensor. For example, the device scrolls through or translates at least a portion of the content displayed in the first user interface to the right when the fingerprint moves to the right on the fingerprint sensor and scrolls through or translates at least a portion of the content displayed in the first user interface to the left when the fingerprint moves to the left on the fingerprint sensor (e.g., as described above with reference to).

622 In some embodiments, displaying the second user interface includes () displaying an animated transition of the second user interface appearing on the display; and the speed of the animated transition is based on a displacement of the fingerprint on the fingerprint sensor. For example, the device displays the first user interface sliding upward on the display to make room for the second user interface that slides onto the display from a bottom of the display.

624 204 169 204 5 FIG.A 5 FIG.A In some embodiments, the device includes () a button (e.g., a physical button, such as home button or menu button,); and the fingerprint sensor is integrated into the button (e.g., fingerprint sensoris integrated into the physical home button,). In some embodiments, the physical button is an intensity-sensitive region of the housing of the device that is determined to have been activated by the device when the intensity detected on the button is increased over an activation intensity threshold. In some embodiments, the physical button is coupled to an electronic switch where movement of the physical button along a first axis (e.g., up and down or left to right) closes the switch and activates the physical button (sometimes resulting in an audible or tactile “click”).

5 FIG.M 5 FIG.M 5 FIG.M 5 FIG.N 626 534 204 204 628 In some embodiments, while the first user interface (e.g., the map application user interface,) is displayed, the device detects () activation of the button (e.g., activationof the physical home button or menu buttonsuch as a tap input or a press input on the physical button,). For example, the device detects a user clicking the button or pressing an intensity-sensitive region corresponding to the button with an intensity above an activation intensity threshold. In response to detecting activation of the button, the device replaces () the first user interface (e.g., the map application user interface,) with a predefined user interface (e.g., a home screen or application launch interface as shown in; or a search application user interface) associated with activating the button.

517 630 532 204 204 632 5 FIG.K 5 FIG.K 5 5 FIGS.K-L In some embodiments, while the second user interface (e.g., multi-tasking user interface,) is displayed, the device detects () activation of the button (e.g., activationof the physical home button or menu buttonsuch as, via a tap or press input on the physical button,). For example, the device detects a user clicking the button or pressing an intensity-sensitive region corresponding to the button with an intensity above an activation intensity threshold. In response to detecting activation of the button, the device ceases () to display the second user interface (e.g., and redisplays or reactivates the first user interface, such as the map application user interface, as described above with reference to).

5 FIG.J 5 FIG.J 5 5 FIGS.J-K 634 530 204 636 517 In some embodiments, while the first user interface (e.g., the map application user interface,) is displayed, the device detects () a double activation of the button (e.g., double activationof the physical home button or menu buttonsuch as, via two successive taps or press inputs occurring in quick succession with less than a predefined interval of time lapse between the two successive taps, for example with an interval of time less than 500 milliseconds, 300 milliseconds, 150 milliseconds, 50 milliseconds or some other reasonable time interval; with or without a finger lift-off between the two successive taps, as described above with reference to). For example, the device detects a user double clicking a button or pressing the intensity-sensitive region corresponding to the button with an intensity above an activation intensity threshold, reducing the intensity below the activation intensity threshold and then pressing the intensity-sensitive region with an intensity above the activation threshold. In response to detecting double activation of the button, the device displays () the second user interface on the display (e.g., displays multitasking user interface, as described above with reference to; or a most recently used application).

517 638 518 520 640 517 520 518 5 FIG.F 5 FIG.F 5 5 FIGS.F-G 5 FIG.G 5 FIG.F In some embodiments, while displaying the second user interface (e.g., multi-tasking user interface,) in accordance with the determination that the movement of the fingerprint is in the second direction, the device detects () movement of the fingerprint on the fingerprint sensor in the first direction (e.g., fingerprintand associated movementin a lateral or horizontal direction or along the short-axis of the multi-purpose device; from right to left,). In response to detecting movement of the fingerprint on the fingerprint sensor in the first direction, the device navigates () through the second user interface (e.g., the device scrolls through representations of a plurality of recently used or currently open applications in accordance with movement of the fingerprint on the fingerprint sensor). For example, as shown in, the device translates at least a portion of the second user interface (e.g., multi-tasking user interface) in the direction of movementof fingerprintto display additional portions of the second user interface (e.g., additional icons including a “messages” icon corresponding to a message application, a “calendar” icon corresponding to a calendar application, and a “clock” icon corresponding to clock application, as shown in, that were not previously displayed in).

In some embodiments, navigating through the second user interface includes navigating through the second user interface at a rate determined in accordance with a displacement of the fingerprint on the fingerprint sensor. For example, the device scrolls through icons corresponding to the plurality of recently used or currently open applications at a rate of five icons/second for each 1 millimeter of movement by the fingerprint on the fingerprint sensor from a starting position of the fingerprint on the fingerprint sensor. In some embodiments, navigating through the second user interface includes navigating through the second user interface by an amount determined in accordance with a displacement of the fingerprint on the fingerprint sensor. For example, the device scrolls through icons corresponding to the plurality of recently used or currently open applications by one icon for each 1 millimeter of movement by the fingerprint on the fingerprint sensor. In some embodiments, navigating through the second user interface includes navigating through the second user interface in a direction determined in accordance with a direction of movement of the fingerprint on the fingerprint sensor. For example, the device scrolls the representations of the plurality of recently used or currently open applications to the right when the fingerprint moves to the right on the fingerprint sensor and scrolls the representations of the plurality of recently used applications to the left when the fingerprint moves to the left on the fingerprint sensor.

517 642 516 514 524 522 644 517 5 FIG.H 5 5 FIGS.D-E 5 5 FIGS.H-I 5 FIG.I 5 FIG.H In some embodiments, while displaying the second user interface (e.g., multi-tasking user interface,) in accordance with the determination that the movement of the fingerprint is in the second direction, the device detects () movement of the fingerprint on the fingerprint sensor in a third direction (e.g., a direction that is substantially opposite to the second direction; such as a vertical direction or along the long-axis of the multi-purpose device; downward). For example, movementof fingerprintin(e.g., movement in the first direction) is upward movement in a vertical direction along the long-axis of the multi-purpose device; while movementof fingerprintin(e.g., movement in the third direction) is downward movement in a vertical direction along the long-axis of the multi-purpose device (e.g., substantially opposite to the second direction). In response to detecting movement of the fingerprint on the fingerprint sensor in the third direction, the device ceases () to display the second user interface. For example, as shown in, the device ceases to display multi-tasking user interfacethat was previously displayed inand redisplays the first user interface (e.g., the map application user interface), including portions of the first user interface that ceased to be displayed when the second user interface was displayed.

6 6 FIGS.A-C 6 6 FIGS.A-C 600 600 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, user interfaces, animated transitions described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, user interfaces, animated transitions described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

7 FIG. 7 FIG. 700 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

7 FIG. 700 702 704 706 708 702 706 708 710 712 714 716 718 As shown in, an electronic deviceincludes a display unitconfigured to display a first user interface, optionally a touch-sensitive surface unit, a fingerprint sensor unit; and a processing unitcoupled to the display unitand the fingerprint sensor unit. In some embodiments, the processing unitincludes a detecting unit, a navigating unit, a display enabling unit, a replacing unit, and a ceasing unit.

708 710 706 712 702 714 The processing unitis configured to: while displaying the first user interface on the display, detect (e.g., with the detecting unit) movement of a fingerprint on the fingerprint sensor; and in response to detecting movement of the fingerprint on the fingerprint sensor: in accordance with a determination that the movement of the fingerprint is in a first direction, navigate through the first user interface (e.g., with the navigating unit); and in accordance with a determination that the movement of the fingerprint is in a second direction different from the first direction, enable display of a second user interface different from the first user interface on the display unit(e.g., with the display enabling unit).

In some embodiments, the first direction is perpendicular to the second direction.

In some embodiments, the first user interface is a user interface of a first application; and the second user interface is a multitasking user interface that includes representations of a plurality of concurrently open applications.

708 706 710 706 712 In some embodiments, the processing unitis further configured to: while enabling display of the second user interface in accordance with the determination that the movement of the fingerprint is in the second direction, detect movement of the fingerprint on the fingerprint sensorin the first direction (e.g., with the detecting unit); and in response to detecting movement of the fingerprint on the fingerprint sensorin the first direction, navigate through the second user interface (e.g., with the navigating unit).

706 710 706 718 In some embodiments, the processing unit is further configured to: while enabling display of the second user interface in accordance with the determination that the movement of the fingerprint is in the second direction, detect movement of the fingerprint on the fingerprint sensorin a third direction (e.g., with the detecting unit); and in response to detecting movement of the fingerprint on the fingerprint sensorin the third direction, cease to enable display of the second user interface (e.g., with the ceasing enabling unit).

706 In some embodiments, navigating through the first user interface includes navigating through the first user interface at a rate determined in accordance with a displacement of the fingerprint on the fingerprint sensor.

706 In some embodiments, navigating through the first user interface includes navigating through the first user interface by an amount determined in accordance with a displacement of the fingerprint on the fingerprint sensor.

706 In some embodiments, navigating through the first user interface includes navigating through the first user interface in a direction determined in accordance with a direction of movement of the fingerprint on the fingerprint sensor.

702 714 706 In some embodiments, enabling display of the second user interface includes enabling display of an animated transition of the second user interface appearing on the display unit(e.g., with the display enabling unit); and the speed of the animated transition is based on a displacement of the fingerprint on the fingerprint sensor.

706 702 In some embodiments, the fingerprint sensoris separate from the display unit.

704 706 704 In some embodiments, the device includes a touch-sensitive surface unitand the fingerprint sensoris separate from the touch-sensitive surface unit.

706 In some embodiments, the device includes a button and the fingerprint sensoris integrated into the button.

708 710 716 In some embodiments, the processing unitis further configured to: while the first user interface is displayed, detect activation of the button (e.g., with the detecting unit); and in response to detecting activation of the button, replace the first user interface with a predefined user interface associated with activating the button (e.g., with the replacing unit).

708 710 718 In some embodiments, the processing unitis further configured to: while the second user interface is displayed, detect activation of the button (e.g., with the detecting unit); and in response to detecting activation of the button, cease to display the second user interface (e.g., with the ceasing unit).

708 710 702 714 In some embodiments, the processing unitis further configured to: while the first user interface is displayed, detect a double activation of the button (e.g., with the detecting unit); and in response to detecting double activation of the button, enable display of the second user interface on the display unit(e.g., with the display enabling unit).

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

6 6 FIGS.A-C 1 1 FIGS.A-B 7 FIG. 1 1 FIGS.A-B 602 604 610 628 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, display operation, detection operation, navigating or displaying operationand replacing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices are configured to display a variety of user interfaces. These user interfaces can include, for example, a home screen, a search screen, and/or user interfaces associated with applications (“apps”) stored on the device. Because the home screen is often the most used user interface, it is desirable for such devices to provide the user with a convenient way to quickly navigate to the home screen. For example, with existing methods and devices, a dedicated “home” button is often provided. However, because of the limitations on the size of some devices (e.g., a portable multifunction device such as a smart phone), providing a dedicated home button requires an undesirable amount of space on the housing of such devices. It is therefore desirable to allow buttons on such devices to serve multiple purposes (e.g., a homing purpose and an application dependent purpose) by providing a convenient method to distinguish between which of the two purposes a user desired to activate.

In the embodiments described below, an improved method for allowing buttons to serve multiple purposes is provided. The buttons in the embodiments described below include integrated fingerprint sensors. When such a button is activated (e.g., pressed) after continuously detecting a fingerprint on an integrated fingerprint sensor for less than a predetermined amount of time (e.g., a quick press of the button), the button performs a first operation (e.g., a home operation). On the other hand, when the button is activated after continuously detecting a fingerprint for more than a predetermined amount of time (e.g., after hovering on the fingerprint sensor), the device performs a second operation (e.g., an application specific operation).

450 451 100 112 167 8 8 9 9 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 169 359 1 359 2 112 450 451 359 2 359 1 359 1 450 112 1 FIG.A 4 FIG.B 4 FIG.B 8 8 9 9 FIGS.A-O andA-C 8 8 FIGS.A-O 8 8 FIGS.A-O 8 8 FIGS.A-O 8 8 FIGS.A-O 8 8 FIGS.K-O In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-O andA-C includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, embodiments described with reference towill be discussed with reference to device operations that are performed in response to detecting inputs described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon touch screen. However, analogous operations are, optionally, performed on a device with a display, a separate touch-sensitive surfaceand a separate fingerprint sensor-with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon display. In some embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. In some embodiments, the focus selector is a user-controlled pointer that can be used to point to a respective user interface object in a plurality of user interface objects (e.g., an object selected by a “scroll-wheel,” as described with reference to). In some embodiments, a visual representation of the focus selector is displayed (e.g., a user interface object to which the focus selector is pointing is visually distinguished from the other user interface objects).

8 8 FIGS.A-D 8 8 FIGS.A-D 8 8 FIGS.G-H 204 204 169 illustrate exemplary user interfaces to be used in conjunction with multi-purpose buttons in accordance with some embodiments. In particular,illustrate an example in which, under certain conditions described herein, activation of a “home button” (e.g., button) causes the device to perform a “home” operation (e.g., a first operation), whereupon the device returns to a home screen (e.g., displays an application-launch interface that includes a plurality of icons representing applications that a user may choose from). In this example, buttonincludes an integrated fingerprint sensor. Under other conditions, as described with reference to, activation of the home button causes the device to perform a second, different operation.

8 FIG.A 100 illustrates an example of the device running an archery video game application on portable multifunction device.

8 FIG.B 8 FIG.B 806 169 806 806 806 204 806 204 204 204 204 806 204 806 204 169 0 0 0 0 0 0 illustrates detection of a fingerprint(in this example, corresponding to a user's right thumb) on integrated fingerprint sensor.also illustrates, respectively, indications of the length of time that fingerprintis continuously detected on fingerprint sensor as well as the force of fingerprinton the button. The respective indications of length of time and force of fingerprinton the button are not typically shown on the device, but instead are illustrated herein for convenience of explanation. In this example, buttonis activated when the force of fingerprintexceeds an activation threshold (AT). In some embodiments, for example when buttonis a physical or mechanical button, the activation threshold ATis a function of spring properties of certain mechanical features of buttonand/or friction between the mechanical features of button. That is to say, in some embodiments, there is a minimum force on the button which causes the button to activate. In some embodiments, activation of buttonoccurs on a “down-stroke” (e.g., when the force of fingerprintis continuously detected from below ATto above AT). In some embodiments, activation of buttonoccurs on an “up-stroke” (e.g., when the force of fingerprintis continuously detected from above ATto below AT). In some other embodiments, for example when buttonis a virtual button, a specific gesture detected by fingerprint sensorcauses activation of the button.

8 8 FIG.C-D 8 FIG.C 8 FIG.D 204 806 204 806 As shown in, when the device detects activation of button(shown in) prior to continuous detection of fingerprintfor an amount of time greater than a predetermined time period PT, the device performs a first operation. In this example, the first operation is a “home operation” (shown in). In some embodiments, the device performs the first operation when the device detects activation of buttonprior to continuous detection of fingerprintfor an amount of time equal or greater than a predetermined time period PT.

8 8 FIGS.D-F 8 8 FIGS.D-F In some embodiments, as explained below with reference to, the first operation is context dependent. For example, in some embodiments, the first operation depends on (e.g., is associated with) a currently displayed user interface. In the previous example in which the currently displayed user interface included a displayed archery video game, the first operation associated with the archery video game returns the device to a home screen.illustrate an example of the first operation when the home screen is the currently displayed user interface.

8 FIG.D 8 FIG.B 8 FIG.D 808 808 806 808 806 169 In addition to illustrating an example of a home screen,also illustrates an example of detection of a second fingerprint. Detection of fingerprintis analogous to detection of fingerprintdescribed with reference to. In some embodiments, fingerprintis a continuation of fingerprint(e.g., in some embodiments, after navigating to the home screen shown, the user need not discontinue and reapply contact with fingerprint sensorin order to make use of the functionality described in the example).

8 FIG.E 8 8 FIGS.C-D 8 FIG.F 204 illustrates activation of buttonprior to a predetermined time period PT elapsing, as previously described with reference to. In this example, because predetermined time period PT has not elapsed, the device performs a first operation associated with a home screen (e.g., a application-launch interface). In this example, the first operation navigates the device to a search screen, as shown in. In some embodiments, the first operation associated the home screen takes the user to an alternate home screen (e.g., the device navigates through a plurality of home screen pages, the plurality of home screens being required because of a large number of applications on the device).

8 8 FIGS.G-J 8 8 FIGS.G-J 8 FIG.G 8 FIG.H 8 8 FIGS.I-J 8 FIG.I 8 8 FIGS.I andJ 204 810 810 810 810 169 802 169 802 204 204 169 204 169 204 illustrate exemplary user interfaces to be used in conjunction with multi-purpose buttons in accordance with some embodiments. In particular,illustrate an example of the device performing a second, different operation upon activation of buttonsubsequent to continuous detection of a fingerprintfor greater than the predetermined time. In, a fingerprintis initially detected. Thus, the indicator showing the length of the continuously detected finger printillustrates that, initially, no time has passed.illustrates movement of fingerprintover fingerprint sensorand corresponding movement of archer(e.g., in the application in this example, fingerprint sensoracts a “joy-stick” controlling the position, or alternatively the change in position, of archerprior to activation of button). In some embodiments, such functionality (e.g., the joystick functionality) becomes operable subsequent to an elapse of the predetermined time, and remains operable until activation of button. In some embodiments, such functionality is operable in the application whenever a fingerprint is detected on fingerprint sensordespite repeated activations of button(e.g., a first-person shooter video game in which fingerprint sensoracts as a joy-stick redirecting a player and activation of the button allows the player to fire his or her weapon).illustrate activation of button(shown in) and the resulting performance of a second operation (e.g., firing an arrow, as shown in), which is different from the first operation (e.g., a home operation).

8 8 FIGS.K-O 8 8 FIGS.K-O 8 8 FIGS.K-O 8 FIG.K 8 8 FIGS.K-O illustrate exemplary user interfaces to be used in conjunction with multi-purpose buttons in accordance with some embodiments. In particular,) illustrate an example in which the user interface includes a plurality of affordances, for example user selectable icons corresponding applications on the home screen (e.g., phone app, browser app, etc.).also illustrate a focus selector corresponding to a currently selected affordance (e.g., the icon representing the phone application inis highlighted, providing a visual cue that the focus selector is “over” the phone application and that the phone application is currently selected, although the device has not yet received a command to run the phone application). Furthermore,illustrate an example of embodiments in which the second operation includes performing an operation associated with application that corresponds to a currently selected affordance of the plurality of affordances.

8 FIG.K 8 FIG.L 8 FIG.M 169 169 812 illustrates an example in which the focus selector's position is controlled by circumferential movement around fingerprint sensor(e.g., the fingerprint sensor acts as a “scroll-wheel” with exemplary properties illustrated below). In this example, clockwise circumferential movement, as shown, moves the focus selector to the right as shown in(movement of the focus selector over the browser app) and(movement of the focus selector over the mail app). Likewise, although not shown, counter-clockwise circumferential movement around fingerprint sensormoves the focus selector in an opposite direction (e.g., to the left). In some embodiments, the plurality of affordances are ordered and circumferential movement of fingerprintincrements the focus selector up or down in the order (e.g., movement of the fingerprint circumscribing an arc of a pre-defined angle, such as 90 degrees, increments the focus selector). In some embodiments, when the focus selector reaches the last affordance in the order, further incrementing the focus selector in the same direction returns the focus selector to the first affordance in the order. In some embodiments, the scroll-wheel functionality is available in an accessibility mode of the device different from a normal mode of the device.

8 FIG.N 8 FIG.O 204 812 812 814 1 814 3 illustrates an example of activation of buttonwhen the force of fingerprintexceeds an activation threshold, as described previously. In this example, because fingerprinthas been continuously detected for predetermined period of time PT, the device performs the second operation. In this example, the second operation depends on the currently selected affordance. For example,shows that the second operation has launched the mail app, as the device is now displaying an email interface including user interface objects-through-corresponding to email messages.

9 9 FIGS.A-C 3 FIG. 1 FIG.A 900 900 300 100 900 are flow diagrams illustrating a methodof allowing a button to serve multiple purposes, in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

900 As described below, the methodprovides an intuitive way to allow a button to serve multiple purposes. The method reduces the cognitive burden on a user when using buttons, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to perform various device operations faster and more efficiently conserves power and increases the time between battery charges.

902 904 906 8 FIG.A The device detects (), at an electronic device with a button that includes an integrated fingerprint sensor, a fingerprint on the integrated fingerprint sensor. In some embodiments, the electronic device includes () a display. In some embodiments, prior to detecting the fingerprint on the integrated fingerprint sensor, the device displays () a first user interface on the display (e.g., the user interface shown in).

908 910 912 8 FIG.K In some embodiments, prior to detecting activation () of the button the device detects () the fingerprint on the integrated fingerprint sensor for more than the predetermined period of time. In response to detecting the fingerprint on the integrated fingerprint sensor for more than the predetermined period of time, the device displays () a representation of a focus selector on the display (e.g., a focus selector representation that was not displayed just prior to detecting the fingerprint, such as the highlighting of the phone app in).

914 916 8 FIG.C 8 8 FIGS.B-D 8 8 FIGS.D-F While continuing to detect the fingerprint on the integrated fingerprint sensor, the device detects () activation of the button (e.g., as shown in). In some embodiments, the physical button is an intensity-sensitive region of the housing of the device that is determined to have been activated by the device when the intensity detected on the button is increased over an activation intensity threshold. In some embodiments, the physical button is coupled to an electronic switch where movement of the physical button along a first axis (e.g., up and down or left to right) closes the switch and activates the physical button (sometimes resulting in an audible or tactile “click”). In response to detecting activation of the button and in accordance with a determination that the fingerprint was not continuously detected on the integrated fingerprint sensor for more than a predetermined period of time prior to detecting activation of the button (e.g., the fingerprint was not continuously detected on the fingerprint sensor for at least 0.3, 0.5, 1, or 2 seconds immediately prior to detecting activation of the button), the device perform () a first operation, where the first operation is a predefined operation associated with activation of the button (e.g., the homing operation described with reference to, or, alternatively, the search screen operation performed with reference to).

918 920 8 8 FIGS.B-D 8 FIG.E 8 FIG.F In some embodiments, the first user interface is a user interface of a first application, and the first operation includes () ceasing to display the first user interface and displaying an application launch interface (e.g., ceasing to display the archery video game,). In some embodiments, the first user interface is an application launch interface, and the first operation includes () ceasing to display the application launch interface () and displaying a predefined application associated with activation of the button (e.g., the search user interface,).

922 8 8 FIGS.G-J In response to detecting activation of the button and in accordance with a determination that the fingerprint was continuously detected on the integrated fingerprint sensor for more than the predetermined period of time prior to detecting activation of the button (e.g., the fingerprint was continuously detected on the fingerprint sensor for at least 0.3, 0.5, 1, or 2 seconds immediately prior to detecting activation of the button), the device performs () a second operation different from the first operation, where the second operation is an operation associated with a user interface displayed on the display immediately prior to detecting activation of the button (e.g., the archery fires his or her bow, as shown in).

924 In some embodiments, the user interface of the first application includes () a plurality of affordances (e.g., menus, controls, hyperlinks and the like) that correspond to respective operations associated with the first application, and the second operation includes performing an operation associated with the first application that corresponds to a currently selected affordance of the plurality of affordances (e.g., the device performs an operation corresponding to an affordance displayed within the application that is currently proximate to or collocated with a focus selector).

920 926 8 FIG.K In some embodiments, the first user interface is an application launch interface (see), the application launch interface includes () a plurality of application-launch affordances (e.g., application icons,) that correspond to respective applications in a plurality of applications, and the second operation includes launching a respective application that corresponds to a currently selected application-launch affordance of the plurality of application-launch affordances (e.g., the device launches an application corresponding to an application icon that is currently proximate to or collocated with a focus selector).

928 In some embodiments, the second operation is dependent () on a location of a focus selector in the first user interface (e.g., the second operation is an operation that corresponds to activation of a particular user interface object in the first user interface, such as activation of a hyperlink in a webpage or launching an application corresponding to an application icon in an application launch interface), and the first operation is independent of the location of the focus selector in the first user interface. For example, the first operation includes displaying a home screen or application launch screen, displaying a predefined application, and/or ceasing to display a user interface corresponding to a currently displayed application. In some embodiments, a representation of the focus selector is displayed in the first user interface and is ignored when performing the first operation. In some embodiments, a representation of the focus selector is not displayed in the first user interface.

930 932 934 942 936 936 938 940 In some embodiments, prior to detecting () activation of the button, the device displays () a representation of a focus selector on the display. The device detects () movement of the fingerprint across the fingerprint sensor on the button. In some embodiments, throughout the movement of the fingerprint across the fingerprint sensor, the fingerprint is continuously detected on the fingerprint sensor. In some embodiments, in response to detecting () the movement of the fingerprint on the fingerprint sensor, the device moves the representation of the focus selector on the display in accordance with movement of the fingerprint (e.g., the device scrolls the focus selector through selectable user interface objects in accordance with a swipe or circular gesture performed with the fingerprint without activating the button). On the other hand, in some embodiments, in response to detecting () the movement of the fingerprint, the device performs () a third operation (e.g., an option switching operation) in accordance with the movement of the fingerprint. In some embodiments, the third operation includes () selecting, as a currently selected option, a respective option from a plurality of options (e.g., selecting an option from a drop down menu or a item from a set of items, and the second operation includes performing an action associated with the currently selected option. In some embodiments, the first operation is independent () of the currently selected option (e.g., committing the currently selected option as a choice for a content field, or using/activating the item). As one example, the option switching switches between items or weapons in a game, and the second operation includes using the item or firing the weapon in the game. In this example, the first operation is, optionally, exiting the game and returning to a home screen or application launch screen of the device.

9 9 FIGS.A-C 9 9 FIGS.A-C 900 900 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, gestures, user interface objects, focus selectors, described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, user interface objects, focus selectors, described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

10 FIG. 10 FIG. 1000 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

10 FIG. 1000 1001 1002 1008 1001 1000 1006 1004 1006 1004 1001 1008 1008 1010 1012 1014 1016 As shown in, an electronic deviceincludes a button unitwith an integrated a fingerprint sensor unit; and a processing unitcoupled to the button unit. The electronic deviceoptionally includes a display unitconfigured to enable the display of one or more user interfaces; and a touch-sensitive surface unitconfigured to receive contacts. In some embodiments, the display unitand the touch-sensitive surface unitare coupled to the button unitand/or the processing unit. In some embodiments, the processing unitincludes a detecting unit, a first operation performing unit, a second operation performing unit, and a display enabling unit.

1008 1010 1002 1002 1008 1001 1001 1002 1001 1008 1012 1001 1001 1002 1001 1008 1014 1006 1001 The processing unitis configured to: detect (e.g., with detecting unit) a fingerprint on the integrated fingerprint sensor unit. While continuing to detect the fingerprint on the integrated fingerprint sensor unit, the processing unitis further configured to detect activation of the button unit. In response to detecting activation of the button unit: in accordance with a determination that the fingerprint was not continuously detected on the integrated fingerprint sensor unitfor more than a predetermined period of time prior to detecting activation of the button unit, the processing unitis configured to perform a first operation (e.g., with first operation performing unit), wherein the first operation is a predefined operation associated with activation of the button unit. In response to detecting activation of the button unit: in accordance with a determination that the fingerprint was continuously detected on the integrated fingerprint sensor unitfor more than the predetermined period of time prior to detecting activation of the button unit, the processing unitis configured to perform a second operation different from the first operation (e.g., with second operation performing unit), wherein the second operation is an operation associated with a user interface displayed on the display unitimmediately prior to detecting activation of the button unit.

1006 1002 1016 In some embodiments, the display unitis configured to, prior to detecting the fingerprint on the integrated fingerprint sensor unit, enable the display of a first user interface (e.g., with display enabling unit).

1006 In some embodiments, the first user interface is a user interface of a first application; and the first operation includes ceasing to enable the display of the first user interface and enabling the display of an application launch interface on the display unit.

In some embodiments, the user interface of the first application includes a plurality of affordances that correspond to respective operations associated with the first application; and the second operation includes performing an operation associated with the first application that corresponds to a currently selected affordance of the plurality of affordances.

1001 In some embodiments, the first user interface is an application launch interface; and the first operation includes ceasing to enable the display of the application launch interface and enabling the display of a predefined application associated with activation of the button unit.

In some embodiments, the application launch interface includes a plurality of application-launch affordances that correspond to respective applications in a plurality of applications; and the second operation includes launching a respective application that corresponds to a currently selected application-launch affordance of the plurality of application-launch affordances.

1008 1001 1002 1002 1006 In some embodiments, the processing unitis further configured to, prior to detecting activation of the button unit: detect the fingerprint on the integrated fingerprint sensor unitfor more than the predetermined period of time; and in response to detecting the fingerprint on the integrated fingerprint sensor unitfor more than the predetermined period of time, enable the display of a representation of a focus selector on the display unit.

1008 1001 1006 1008 1002 1006 In some embodiments, the processing unitis further configured to, prior to detecting activation of the button unit: display a representation of a focus selector on the display unit; detect movement of the fingerprint on the fingerprint sensor unit; and in response to detecting the movement of the fingerprint on the fingerprint sensor unit, move the representation of the focus selector on the display unitin accordance with movement of the fingerprint.

In some embodiments, the second operation is dependent on a location of a focus selector in the first user interface; and the first operation is independent of the location of the focus selector in the first user interface.

1008 1001 1006 1002 1002 1006 In some embodiments, the processing unitis further configured to, prior to detecting activation of the button unit: display a representation of a focus selector on the display unit; detect movement of the fingerprint on the fingerprint sensor unit; and in response to detecting the movement of the fingerprint on the fingerprint sensor unit, move the representation of the focus selector on the display unitin accordance with movement of the fingerprint.

In some embodiments, the third operation includes selecting, as a currently selected option, a respective option from a plurality of options; and the second operation includes performing an action associated with the currently selected option.

In some embodiments, the first operation is independent of the currently selected option.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

9 9 FIGS.A-C 1 1 FIGS.A-B 10 FIG. 1 1 FIGS.A-B 901 906 916 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detecting operation, displaying operation, and performing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have different orientations, such as a portrait orientation and a landscape orientation. When these devices display user interfaces and content, the user interfaces and content is typically displayed in the same orientation as the device orientation. Sometimes, the user may want to have user interfaces and content displayed in an orientation different from the device orientation. The user can force the device to display user interfaces and content in a particular orientation by activating an option to lock the orientation of the user interfaces and content. In some methods, activating the orientation locking option requires opening menus and sub-menus and/or multiple gestures and button presses. The embodiments described below improve on these methods by allowing a user to lock the orientation with a simpler input. On a device with a fingerprint sensor, the user locks the user interface and content orientation by rotating a fingerprint in one direction on a fingerprint sensor while the user interface or content is displayed in the desired orientation, and unlock the orientation by rotating the fingerprint in the opposite direction on the fingerprint sensor. This makes the orientation locking process more efficient and simple for users.

450 451 100 112 167 11 11 12 12 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 112 450 451 359 1 359 2 359 1 359 2 450 1 FIG.A 4 FIG.B 4 FIG.B 11 11 12 12 FIGS.A-F andA-B 11 11 FIGS.A-F 11 11 FIGS.A-F In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-F andA-B includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated into the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screenand fingerprint sensor; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and an integrated fingerprint sensor-or a separate fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-or the separate fingerprint sensor-while displaying the user interfaces shown inon the display.

11 FIG.A 11 FIG.A 11 FIG.A 11 FIG.A 1102 112 100 1102 100 1102 100 1100 112 1101 112 112 1100 1102 112 1102 1100 1101 illustrates user interfacedisplayed on touch screenof a device (e.g., device). User interfaceis a user interface of, for example, an application or an operating system running on device. For example, user interfaceas shown inis a user interface for a notes application. In, deviceis oriented such that the vertical height (i.e., the dimension indicated by arrow) of touch screenis longer than the horizontal width (i.e., the dimension indicated by arrow); touch screenis in portrait orientation (e.g., a long axis of touch screenis parallel to an up/down axis indicated by arrow). In, user interfaceis displayed in portrait mode, in accordance with the portrait orientation of touch screen. For example, in portrait mode, user interface, right side up, is longer on dimensionthan dimension.

11 FIG.B 11 FIG.B 100 1100 112 1101 112 112 1101 1102 112 1102 1101 1100 In, deviceis oriented such that the vertical height (i.e., dimension) of touch screenis shorter than the horizontal width (i.e., dimension); touch screenis in landscape orientation (e.g., a long axis of touch screenis parallel to a right/left axis indicated by arrow). In, user interfaceis displayed in landscape mode, in accordance with the landscape orientation of touch screen. For example, in landscape mode, user interface, right side up, is longer on dimensionthan dimension.

11 11 FIGS.A-F 100 112 112 100 112 100 100 112 112 100 It should be appreciated that, in, the longer dimension of deviceand the longer dimension of touch screenare parallel. Thus, touch screenis in portrait orientation (i.e., vertical height longer than horizontal width) when deviceis in portrait orientation. Analogously, touch screenis in landscape orientation when deviceis in landscape orientation. In some embodiments, the longer dimension of deviceis perpendicular to the longer dimension of touch screen; touch screenis in portrait orientation when deviceis in landscape orientation, and vice versa.

11 11 FIGS.A-B 11 FIG.A 11 FIG.B 11 FIG.A 11 FIG.B 11 FIG.B 11 FIG.A 11 FIG.B 11 FIG.A 1102 112 112 100 112 1102 1102 100 100 1102 100 100 1102 As shown in, user interfaceis displayed in portrait or landscape mode in accordance with the orientation of touch screen. When touch screenchanges orientation (e.g., by a user rotating devicearound an axis perpendicular to the surface of touch screen) and user interfaceis not locked to a particular orientation-specific mode, user interfaceis displayed in a mode in accordance with the new orientation (e.g., based on a sensed direction of gravity from an accelerometer integrated into the device). For example, when deviceis turned from portrait orientation, as in, to landscape orientation, as in, devicechanges the displayed user interfacefrom portrait mode, as in, to landscape mode, as in. Similarly, when deviceis turned from landscape orientation, as in, to portrait orientation, as in, devicechanges the displayed user interfacefrom landscape mode, as in, to portrait mode, as in.

11 FIG.C 11 11 FIG.C-D 100 1104 169 1104 169 169 100 1104 169 1104 1102 1104 1104 169 1104 1104 169 illustrates devicedetecting fingerprinton fingerprint sensor. Fingerprintis placed on fingerprint sensorby, for example, a user making contact on fingerprint sensorwith a finger.illustrates devicedetecting a rotation of fingerprintin a clockwise direction around fingerprint sensor. The rotation of fingerprintis detected while user interfaceis displayed in portrait mode. In some embodiments, the rotation of fingerprintis a twisting of fingerprinton fingerprint sensor. In some other embodiments, the rotation of fingerprintis a revolution of fingerprintaround fingerprint sensor.

1104 169 1102 100 1102 1106 1102 1102 1102 112 1102 100 1102 1102 100 1102 1102 100 11 FIG.D 11 FIG.E In response to detecting the rotation of fingerprintclockwise around fingerprint sensorwhile user interfaceis displayed in portrait mode, devicelocks user interfaceinto portrait mode. In some embodiments, a visual indication (e.g., icon,) that user interfaceis locked in portrait mode is displayed. While user interfaceis locked in portrait mode, user interfaceis displayed in portrait mode whether touch screenis in portrait orientation or in landscape orientation. When user interface, locked in portrait mode, is displayed and deviceis rotated to landscape orientation, user interfaceremains in portrait mode; user interfaceis displayed as if rotated 90 degrees. For example,shows devicein landscape orientation while user interfaceis locked in portrait mode; user interfaceis displayed in portrait mode despite the landscape orientation of device.

1102 100 1108 1108 169 1108 169 1102 1102 1106 1102 1102 1106 100 1108 169 1102 1102 100 1102 1102 100 11 11 FIGS.E-F 11 FIG.D 11 FIG.F 11 11 FIGS.A-B 11 FIG.F 11 FIG.B While user interfaceis locked in portrait mode, the devicedetects fingerprintand a rotation of fingerprintin a counter-clockwise direction around fingerprint sensor, as shown in. In response to detecting the rotation of fingerprintcounterclockwise around fingerprint sensorwhile user interfaceis locked in portrait mode, user interfaceis unlocked from portrait mode. If a visual indication (e.g., icon,) that user interfacewas locked in portrait mode was displayed while user interfacewas locked, then the visual indication ceases to be displayed. For example,shows iconceasing to be displayed in response to devicedetecting the rotation of fingerprintcounter-clockwise around fingerprint sensor. After user interfaceis unlocked from portrait mode, the display mode of user interfaceagain follows the orientation of device(e.g., in accordance with a direction of gravity as detected by the device using an accelerometer or other sensor), as in. For example, in, after user interfaceis unlocked from portrait mode, user interfacereverts back to landscape mode, as in, in accordance with the landscape orientation of device.

1102 1104 1102 100 1102 1108 1102 1102 11 11 FIGS.C-D 11 11 FIGS.E-F In some embodiments, operations analogous to those described above are performed to lock user interfaceinto landscape mode. For example, a fingerprint rotation analogous to the rotation of fingerprint() while user interfaceis displayed in landscape mode (e.g., in accordance with devicebeing in landscape orientation) locks user interfaceinto landscape mode. A fingerprint rotation analogous to the rotation of fingerprint() while user interfaceis locked in landscape mode unlocks user interfacefrom landscape mode.

1102 In some embodiments, the fingerprint rotation that locks user interfaceinto a particular orientation mode (i.e., portrait mode or landscape mode) is counterclockwise, and the rotation to unlock is clockwise, as opposed to the vice versa described above.

12 12 FIGS.A-B 3 FIG. 1 FIG.A 1200 1200 300 100 1200 are flow diagrams illustrating a methodof locking an orientation of user interface in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

1200 As described below, the methodprovides an intuitive way to lock an orientation of user interface. The method reduces the cognitive burden on a user when locking an orientation of user interface, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to lock an orientation of user interface faster and more efficiently conserves power and increases the time between battery charges.

1202 1102 112 1102 100 100 11 11 FIGS.A-B 11 FIG.A 11 FIG.B The device displays () a user interface on the display, where the user interface has a first orientation-specific mode of operation (e.g., portrait mode) associated with a first orientation of the device and a second orientation-specific mode of operation (e.g., landscape mode) that is associated with a second orientation of the device that is different from the first orientation of the device.shows, for example, user interfacedisplayed on touch screen. User interfaceis displayed in portrait mode when deviceis in portrait orientation (), and displayed in landscape mode when deviceis in landscape orientation ().

1204 1102 11 11 FIGS.A-B In some embodiments, the first orientation-specific mode of operation is a portrait orientation mode of operation, and the second orientation-specific mode of operation is a landscape orientation mode of operation (). As shown in, user interfaceis displayed in portrait mode or landscape mode.

1206 1208 1210 1102 1102 100 100 11 11 FIGS.A-B 11 FIG.A 11 FIG.B While the orientation of the user interface is not locked (), the device displays () the user interface in the first orientation-specific mode of operation when the device is in the first orientation, and displays () the user interface in the second orientation-specific mode of operation when the device is in the second orientation.show user interfacethat is not locked in a particular orientation mode. User interfaceis displayed in portrait mode when deviceis in portrait orientation (), and displayed in landscape mode when deviceis in landscape orientation ().

1212 1104 169 1108 169 1214 169 112 11 FIG.C 11 FIG.E The device detects () a fingerprint on the fingerprint sensor., for example, shows fingerprintdetected on fingerprint sensor. As another example,shows fingerprintdetected on fingerprint sensor. In some embodiments, the fingerprint sensor is () separate from the display. For example, fingerprint sensoris separate from touch screen.

1216 100 1104 169 1218 1220 1104 1102 100 1102 11 FIG.D 11 FIG.D 11 FIG.E While the user interface is in the first orientation-specific mode of operation, the device detects () rotation of the fingerprint on the fingerprint sensor (e.g., where an orientation of the fingerprint changes relative to an orientation of the fingerprint sensor) in a first direction (e.g., clockwise). For example,shows devicedetecting rotation (e.g., twisting) of fingerprinton fingerprint sensorin a clockwise direction. In response to detecting rotation of the fingerprint on the fingerprint sensor in the first direction (), the device locks () the user interface in the first orientation-specific mode of operation. In, for example, in response to detecting the clockwise rotation of fingerprintwhile user interfaceis displayed in portrait mode, devicelocks user interfaceinto portrait mode (e.g., as shown in).

1218 1222 1104 1102 100 1106 1102 11 FIG.D In some embodiments, in response to detecting rotation of the fingerprint on the fingerprint sensor in the first direction (), the device displays () a visual indication that the user interface orientation is locked in the first orientation-specific mode of operation. In, for example, in response to detecting the clockwise rotation of fingerprint, in addition to locking user interfaceinto portrait mode, devicedisplays iconto indicate that user interfaceis locked in portrait mode.

1224 1226 100 1102 100 100 1102 11 FIG.D 11 FIG.E While the user interface is locked in the first orientation-specific mode of operation (), the device detects that the device is in the second orientation and maintains () the user interface in the first orientation-specific mode of operation. For example,shows devicein portrait orientation and user interfacelocked in portrait mode. When deviceis rotated (e.g., by the user) to landscape orientation, devicedetects that it is now in landscape orientation and maintains user interface, which was locked in portrait mode, in portrait mode, as shown in.

1102 100 In some embodiments, while the user interface is in the second orientation-specific mode of operation, the device detects rotation of the fingerprint on the fingerprint sensor (e.g., where an orientation of the fingerprint changes relative to an orientation of the fingerprint sensor) in a first direction (e.g., clockwise); in response to detecting rotation of the fingerprint on the fingerprint sensor in the first direction, the device locks the user interface in the second orientation-specific mode of operation; and while the user interface is locked in the second orientation-specific mode of operation, the device detects that the device is in the first orientation and maintains the user interface in the second orientation-specific mode of operation. Thus, analogously, user interfaceis, optionally, locked into landscape mode, and is displayed in landscape mode even when deviceis in portrait orientation.

1224 1228 1230 100 1108 1102 1108 1102 1102 11 FIG.F In some embodiments, while the user interface is locked in the first orientation-specific mode of operation (), the device detects () rotation of a fingerprint on the fingerprint sensor in a second direction (e.g., counterclockwise) different from the first direction. In response to detecting the rotation of the fingerprint in the second direction, the device unlocks () the user interface from the first orientation-specific mode of operation. For example,shows devicedetecting a counterclockwise rotation (e.g., twisting) of fingerprintwhile user interfaceis locked in portrait mode. In response to detecting the counterclockwise rotation of fingerprintwhile user interfaceis locked in portrait mode, user interfaceis unlocked from portrait mode.

12 12 FIGS.A-B 12 12 FIGS.A-B 1200 1200 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints and user interfaces described above with reference to methodoptionally have one or more of the characteristics of the fingerprints and user interfaces described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

13 FIG. 13 FIG. 1300 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

13 FIG. 1300 1302 1300 1300 1300 1304 1306 1308 1302 1304 1306 1308 1310 1312 1314 1316 1318 As shown in, an electronic deviceincludes a display unitconfigured to display a user interface, where the user interface has a first orientation-specific mode of operation associated with a first orientation of the deviceand a second orientation-specific mode of operation that is associated with a second orientation of the devicethat is different from the first orientation of the device, optionally, a touch-sensitive surface unit, a fingerprint sensor unit; and a processing unitcoupled to the display unit, the touch-sensitive surface unitand the fingerprint sensor unit. In some embodiments, the processing unitincludes a display enabling unit, a detecting unit, a locking unit, a maintaining unit, and an unlocking unit.

1308 1300 1310 1300 1310 1306 1312 1306 1312 1306 1314 1300 1312 1316 The processing unitis configured to: while the orientation of the user interface is not locked: enable display of the user interface in the first orientation-specific mode of operation when the deviceis in the first orientation (e.g., with the display enabling unit), and enable display of the user interface in the second orientation-specific mode of operation when the deviceis in the second orientation (e.g., with the display enabling unit); detect a fingerprint on the fingerprint sensor unit(e.g., with the detecting unit); while the user interface is in the first orientation-specific mode of operation, detect rotation of the fingerprint on the fingerprint sensor unitin a first direction (e.g., with the detecting unit); in response to detecting rotation of the fingerprint on the fingerprint sensor unitin the first direction, lock the user interface in the first orientation-specific mode of operation (e.g., with the locking unit); and while the user interface is locked in the first orientation-specific mode of operation, detect that the deviceis in the second orientation (e.g., with the detecting unit) and maintain the user interface in the first orientation-specific mode of operation (e.g., with the maintaining unit).

1308 1306 1312 1318 In some embodiments, the processing unitis configured to: while the user interface is locked in the first orientation-specific mode of operation, detect rotation of a fingerprint on the fingerprint sensor unitin a second direction different from the first direction (e.g., with the detecting unit); and in response to detecting the rotation of the fingerprint in the second direction, unlock the user interface from the first orientation-specific mode of operation (e.g., with the unlocking unit).

1308 1306 1310 In some embodiments, the processing unitis configured to: in response to detecting rotation of the fingerprint on the fingerprint sensor unitin the first direction, enable display of a visual indication that the user interface orientation is locked in the first orientation-specific mode of operation (e.g., with the display enabling unit).

In some embodiments, the first orientation-specific mode of operation is a portrait orientation mode of operation, and the second orientation-specific mode of operation is a landscape orientation mode of operation.

1306 1302 In some embodiments, the fingerprint sensor unitis separate from the display unit.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

12 12 FIGS.A-B 1 1 FIGS.A-B 13 FIG. 1 1 FIGS.A-B 1212 1216 1220 1226 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operationsand, locking operation, and detecting and maintaining operationsare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices enable users to record media, including recording audio. While recording, a user typically wishes to reduce/cancel noise coming from sources other than the intended source in the recorded audio. For example, a user may wish to reduce the sound of his breathing in the recorded audio. However, a user may wish to stop or toggle the noise reduction at will. For example, the user may wish to provide commentary during certain parts of recorded media while cancelling inadvertent user sounds during other parts. Existing devices either do not provide the user this functionality or require the user to navigate through various menus and user interfaces. In the embodiments described below, an improved method for controlling noise reduction in recorded audio is achieved by using a fingerprint sensor to allow the user to toggle noise reduction on and off. This method streamlines the process of controlling noise reduction in recorded audio by doing so in response to a fingerprint sensor on an electronic device.

450 451 100 112 167 14 14 15 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 450 451 359 1 451 359 2 359 2 450 112 169 359 1 359 2 112 112 1 FIG.A 4 FIG.B 4 FIG.B 14 14 15 FIGS.A-D and 14 14 FIGS.A-D 14 14 FIGS.A-D 14 14 FIGS.A-D 14 14 FIGS.A-D In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-D andincludes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to display, a touch-sensitive surface, and an integrated fingerprint sensor-, however analogous operations are, optionally, performed on a device with a separate touch-sensitive surfaceand a separate fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display. Additionally, analogous operations are, optionally, performed on a device with a touch screenin response to detecting the contacts described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon the touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen.

14 14 FIGS.A-C 14 FIG.A 14 FIG.A 14 FIG.A 14 FIG.B 14 FIG.C 1402 1406 1408 1410 450 100 451 359 1 1404 451 359 1 1408 1404 451 359 1 1408 1404 359 1 1408 illustrate an example of controlling noise reduction in recorded audio with a fingerprint sensor.shows user interface, including media recording interfacewith noise reduction statusand video preview, displayed on displayof a device (e.g., portable multifunction device).further illustrates touch-sensitive surfacewith integrated fingerprint sensor-.also shows the device detecting fingerprint(e.g., a right thumb contact) on touch-sensitive surfaceaway from integrated fingerprint sensor-and noise reduction statusindicating that noise reduction is on (e.g., active) in the recorded audio.shows the device detecting fingerprinton touch-sensitive surfaceover integrated fingerprint sensor-and noise reduction statusindicating that noise reduction is off (e.g., inactive) in the recorded audio.shows the device detecting that fingerprintis no longer on integrated fingerprint sensor-and noise reduction statusindicating that noise reduction is on in the recorded audio.

14 FIG.D 14 FIG.D 1412 1418 1422 1414 1420 1416 illustrates an example of a device with a display, a fingerprint sensor, and a camera sensor for use in controlling noise reduction in recorded audio.shows devicewith fingerprint sensorand displayon first sideand camera sensoron second side.

15 FIG. 3 FIG. 1 FIG.A 1500 1500 300 100 1500 is a flow diagram illustrating a methodof controlling noise reduction in recorded audio with a fingerprint sensor in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

1500 As described below, the methodprovides an intuitive way to control noise reduction in recorded audio with a fingerprint sensor. The method reduces the cognitive burden on a user when controlling noise reduction in recorded audio, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to control noise reduction in recorded audio faster and more efficiently conserves power and increases the time between battery charges.

1502 1406 14 FIG.A The device records () media, where recording the media includes recording audio. For example,shows a device recording media as indicated by media recording interface.

1504 1506 1408 14 FIG.A While recording the media (), the device reduces (), in the recorded audio, noise occurring on a first side of the device. For example, in, noise reduction statusindicates that the device is reducing noise in the recorded audio.

1504 1508 1404 359 1 14 FIG.B While continuing to record the media (), the device detects () a fingerprint on the fingerprint sensor. For example,shows the device detecting fingerprint(e.g., a right thumb contact) on integrated fingerprint sensor-.

1510 1418 1414 1412 14 FIG.D In some embodiments, the fingerprint sensor is located () on the first side of the device. For example,shows fingerprint sensoron first sideof device.

1504 1512 1404 359 1 1408 14 FIG.B While continuing to record the media () and in response to detecting the fingerprint on the fingerprint sensor, the device ceases () to reduce noise occurring on the first side of the device. For example,shows the device detecting fingerprint(e.g., a right thumb contact) on integrated fingerprint sensor-and noise reduction statusindicating that the device has ceased to reduce noise in the recorded audio.

1514 1516 1404 359 1 1404 359 1 1408 14 FIG.C 14 FIG.C In some embodiments, after ceasing to reduce the noise occurring on the first side of the device, the device detects () when the fingerprint is no longer on the fingerprint sensor, and, in response to detecting that the fingerprint is no longer on the fingerprint sensor, the device resumes () reduction of noise occurring on the first side of the device in the recorded audio. For example, a device is initially cancelling noise from one side of the device (e.g., so as to avoid recording the breathing or other noises created by a videographer that the videographer does not intend to record) and subsequently, when the device detects a fingerprint on a fingerprint sensor, the device temporarily ceases to reduce/cancel noise on the first side of the camera (e.g., so that a videographer can comment on the video) and then the device resumes noise cancellation for noises on the first side of the device after or in response to ceasing to detect the fingerprint on the fingerprint sensor (e.g., once the videographer has finished commenting on the video)., for example, shows the device no longer detecting fingerprinton integrated fingerprint sensor-(e.g., fingerprintlifted off or moved away from integrated fingerprint sensor-).further shows noise reduction statusindicating that the device has resumed reduction of noise in the recorded audio.

1518 1420 1416 1412 14 FIG.D In some embodiments, recording the media includes recording () video from a camera sensor on a second side of the device that is different (e.g., opposite from) from the first side of the device. For example, the video is recorded using a front-facing camera and noise is reduced/cancelled using acoustic beam forming to cancel audio input from behind the device. For example,shows camera sensoron second sideof device.

1520 1410 450 14 FIG.A In some embodiments, recording the media includes displaying () a preview of the video on a display located on the first side of the device (e.g., the side that the user is on). From another viewpoint, the method includes displaying a preview of the video on the display located on the first side of the device. For example,shows video previewon displayof a device.

1522 1404 359 1 1408 14 FIG.B 14 FIG.B In some embodiments, recording the media includes recording sounds occurring on a second side of the device that is different from the first side of the device, and, in response to detecting the fingerprint on the fingerprint sensor, the device reduces (), in the recorded audio, noise occurring on the second side of the device. For example,shows the device detecting fingerprint(e.g., a right thumb contact) on integrated fingerprint sensor-. In this example, and in accordance with these embodiments, the device has ceased to reduce noise occurring on the first side of the device (as indicated by noise reduction status) and is reducing noise occurring on the second side of the device (indication of noise reduction on the second side of the device is not shown in).

15 FIG. 15 FIG. 1500 1500 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints and contacts described above with reference to methodoptionally have one or more of the characteristics of the fingerprints and contacts described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

16 FIG. 16 FIG. 1600 In accordance with some embodiments,shows a functional block diagram of electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

16 FIG. 1600 1606 1608 1606 1600 1602 1604 1608 1608 1610 1612 1614 1616 As shown in, electronic deviceincludes fingerprint sensor unitand processing unitcoupled to fingerprint sensor unit. In some embodiments, electronic devicefurther includes display unitand camera sensor unitcoupled to processing unit. In some embodiments, processing unitincludes recording unit, noise reducing unit, detection unit, and display enabling unit.

1608 1610 1608 1612 1614 1612 Processing unitis configured to record (e.g., with recording unit) media with the device, where recording the media includes recording audio. Processing unitis further configured to, while recording the media, reduce (e.g., with noise reducing unit), in the recorded audio, noise occurring on a first side of the device, detect (e.g., with detecting unit) a fingerprint on the fingerprint sensor unit, and, in response to detecting the fingerprint on the fingerprint sensor unit, cease to reduce (e.g., with noise reducing unit) noise occurring on the first side of the device.

1606 In some embodiments, fingerprint sensor unitis located on the first side of the device.

1608 1612 1614 1614 1612 In some embodiments, processing unitis further configured to, after ceasing to reduce (e.g., with noise reducing unit) the noise occurring on the first side of the device, detect (e.g., with detecting unit) when the fingerprint is no longer on the fingerprint sensor, and, in response to detecting (e.g., with detecting unit) that the fingerprint is no longer on the fingerprint sensor unit, resume reduction (e.g., with noise reducing unit) of noise occurring on the first side of the device in the recorded audio.

1600 1608 1610 In some embodiments, electronic devicefurther comprises a camera sensor unit on a second side of the device that is different from the first side of the device and processing unitis further configured to record (e.g., with recording unit) video from the camera sensor unit.

1602 1610 1616 In some embodiments, the electronic device further comprises display unitlocated on the first side of the device and recording (e.g., with recording unit) the media includes enabling display (e.g., with display enabling unit) of a preview of the video on the display unit.

1610 1608 1612 In some embodiments, recording the media includes recording (e.g., with recording unit) sounds occurring on a second side of the device that is different from the first side of the device and processing unitis further configured to, in response to detecting the fingerprint on the fingerprint sensor unit, reduce (e.g., with noise reducing unit), in the recorded audio, noise occurring on the second side of the device.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

15 FIG. 1 1 FIGS.A-B 16 FIG. 1 1 FIGS.A-B 1402 1506 1508 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, recording operation, reducing operation, and detection operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have multiple functions and multiple users. Not all functions are suitable for all users, and thus certain function restrictions are implemented for some user. In some situations, user identification at a device is done through a login process, where users identify themselves specifically with a username and password. However, switching between users through a login process is time-consuming and inefficient. Also, the identity of the specific user is not essential for all function restrictions. The embodiments described below improve on these methods by restricting or modifying functions based on user classes identified by fingerprint. When a device detects a fingerprint of a user, the device identifies a one or more user classes based on the detected fingerprint selects profiles associated with the identified user classes as active, including at least one class that is not unique to the user. When the device receives a request to perform one or more operations, the device performs a respective operation based on the request and the active profiles. The respective operation can be in addition to or instead of the requested operation(s). By customizing operations and functions on the device based on user classes that not unique (e.g., are agnostic) to the specific user identity, function restrictions can be implemented on the device for situations where frequent user login and logoff is inefficient (e.g., parent sharing a device with a child) or unfeasible (e.g., public or semi-public devices for use by a broader population than a closed circle of users).

450 451 100 112 167 17 17 18 18 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 450 451 359 2 359 1 359 1 450 112 169 359 1 359 2 112 112 1 FIG.A 4 FIG.B 4 FIG.B 17 17 18 18 FIGS.A-K andA-B 17 17 FIGS.A-K 17 17 FIGS.A-K 17 17 FIGS.A-K 17 17 FIGS.A-K In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-K andA-B includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to display, a separate touch-sensitive surfaceand a separate fingerprint sensor-, however analogous operations are, optionally, performed on a device with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display. Additionally, analogous operations are, optionally, performed on a device with a touch screenin response to detecting the contacts described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon the touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor.

17 FIG.A 17 FIG.A 4 FIG.A 17 FIG.C 450 451 359 1 1700 450 1700 1700 1700 1704 400 1700 451 1708 1704 451 451 1705 1704 illustrates a device with display, touch-sensitive surface, and integrated fingerprint sensor-. User interfaceis displayed on display. As shown in, user interfaceis, for example, a desktop graphical user interface, a home screen or menu, or the like. User interfaceis, optionally, another type of user interface, such as a user interface for an application. User interfaceoptionally includes one or more application icons, such as “Settings” iconand application icons similar to the application icons displayed in user interface(). A user can interact with user interfaceand make requests for one or more operations to be performed by the device using a respective input element, such as touch-sensitive surface, a button, a mouse, or a keyboard. For example, a user can activate a user interface for adjusting device settings (e.g., settings user interface,) by activating “Settings” iconusing, for example, touch-sensitive surface(e.g., performing a tap gesture on touch-sensitive surfaceassociated with a focus selector such as cursorthat is at a location on the display corresponding to “Settings” icon).

102 370 359 In some embodiments, the device is associated with multiple profiles. The multiple profiles include one or more user-specific profiles and one or more class-based profiles. In some embodiments, each user-specific profile corresponds to a respective user. In some embodiments, each of the class-based profiles, which are distinct from the user-specific profiles, corresponds to a respective class of users. In some embodiments, one or more of the class-based profiles correspond to age-based user classes. Examples of age-based class-based profiles include a class-based profile corresponding to adults (e.g., users 18 years or older), a class-based profile corresponding to non-adults or children (e.g., users under 18 years old), and a class-based profile corresponding to seniors (e.g., users older than 65 years old). In some embodiments, the profiles are retrieved from the memory of the device (e.g., memoryor). In some other embodiments, the profiles are received from a remote computer system. The profiles are retrieved or received in response to one or more particular operations (e.g., detection of a fingerprint on fingerprint sensor) or as part of a predefined routine.

102 370 The device also includes data regarding one or more fingerprints that have been previously registered at the device. Each registered fingerprint corresponds to a respective user-specific profile. In some embodiments, the registered fingerprints are stored in, and retrieved from, the memory (e.g., memoryor) of the device.

1700 450 359 1 359 1 1702 359 1 1706 359 1 1702 1702 1706 1702 1706 1702 17 17 FIGS.A-B 17 FIG.A 17 FIG.B While a user interface (e.g., user interface) is displayed on display, a fingerprint is detected on integrated fingerprint sensor-.illustrate fingerprints of different sizes detected on integrated fingerprint sensor-.illustrates fingerprintbeing detected on integrated fingerprint sensor-, andillustrates fingerprintbeing detected on integrated fingerprint sensor-instead of fingerprint. Fingerprintand fingerprintare fingerprints from two different users who respectively belong to different classes of users. For example, fingerprintis determined to belong to a user in a class of adult users, and fingerprint, which is smaller than fingerprintand determined by the device to be about the size of a child's fingerprint, is determined to belong to a user in a class of child users.

1702 1702 1706 1706 Based on the detected fingerprint, the device selects one or more class-based profiles, from class-based profiles associated with the device, as active profiles that correspond to the detected fingerprint. For example, for fingerprint, the device selects an “Adults” profile as an active profile, as well as any other class-based profile for which the device determines that the user to which fingerprintbelongs fits. For fingerprint, the device selects a “Children” profile, as well as any other class-based profile for which the device determines that the user to which fingerprintbelongs fits. It should be appreciated that a user can be associated with multiple classes. Thus, for a given detected fingerprint, multiple class-based profiles can be selected as active (e.g., a fingerprint could be associated with the “adults” class of users and the “male” class of users or the “children” class of users and the “female” class of users).

1702 1706 1706 1702 In some embodiments, the class-based profiles are selected without regard to authentication of the detected fingerprint. For example, selection of class-based profiles does not require comparison of the detected fingerprint to previously registered fingerprints associated with the device. In some embodiments, the selection of class-based profiles is based on the characteristics and features of the detected fingerprint. For example, the “Adults” profile is selected for fingerprintand the “Children” profile is selected for fingerprintbased on the sizes of these respective fingerprints; fingerprintis determined to be a child's fingerprint based on its size, and fingerprintis determined to be an adult's fingerprint based on its size.

In some embodiments, the class-based profiles are selected in addition to selection of a user-specific profile based on a comparison of the detected fingerprint to previously registered fingerprints at the device. In some embodiments, identification or selection of a user-specific profile aids in the selection of class-based profiles. For example, one or more class-based profiles are, optionally, selected based on a comparison of the user-specific profile to characteristics of the classes based on which the respective class-based profiles are defined.

1704 446 451 1705 1704 1708 1708 1708 1702 1708 1708 17 FIG.C 17 17 FIGS.A-B 17 FIG.A 17 FIG.C After the device selects one or more class-based profiles as active profiles for the device, the device receives an input, with the respective input element, corresponding to a request to perform one or more operations at a device. For example, the device receives an input to activate “Settings” icon(or “Settings” icon) (e.g., a gesture on touch-sensitive surfacewhile a focus selector such as cursoris located over “Settings” icon), which corresponds to a request to display settings user interface(). Depending on the active class-based profiles, settings user interfaceis displayed or an error message is displayed instead. For example, on the device as shown in, settings user interfaceis not displayed if the “Children” profile is active. Thus, for example, if the detected fingerprint is fingerprint(), and thus the “Adults” profile, but not the “Children” profile, is selected as active, settings user interfaceis displayed, as shown in; the operation of displaying settings user interfaceis performed based on the request and the active “Adults” profile.

1706 1708 1710 1708 1710 17 FIG.B 17 FIG.D On the other hand, if the detected fingerprint is fingerprint(), and thus the “Children” profile, but not the “Adults” profile, is selected as active, then settings user interfaceis not displayed. Instead, an error message or the like (e.g., error message) indicating that settings user interfaceis not accessible is displayed, as shown in; the operation of displaying error messageis performed based on the request and the active “Children” profile.

451 432 155 451 112 450 422 400 152 4 FIG.A 1 FIG.A 4 FIG.A 1 FIG.A Another example of an input corresponding to a request to perform one or more operations at a device is a request to open a video application. For example, the user performs a gesture (e.g., a tap or double tap gesture) on touch-surface surfacewhile a focus selector is located over online video icon() to open an online video application (e.g., online video module,). As another example, the user performs a gesture (e.g., a tap or double tap gesture) on touch-sensitive surfacewhile a focus selector (e.g., a contact detected on touchscreenor a cursor displayed on display) is located over a media or video application icon (e.g., icon,) in user interfaceto open an application (e.g., video and music player module,) for viewing videos stored at the device. For convenience, both the application for viewing online videos and the application for viewing videos stored at the device are, hereinafter, both referred to as video application(s).

1712 1712 1714 1714 1712 1714 1714 1712 1714 1 1714 2 1714 3 1714 4 1714 1714 1712 1714 1 1714 2 1714 3 1714 4 17 17 FIGS.E-F 17 FIG.E 17 FIG.E When a video application is opened, a user interface corresponding to the video application, such as user interface, is displayed. User interfaceshows a list of videosstored at the device that can be selected for playback.illustrates differences in the list of videosdisplayed in user interfacedepending on which class-based profile is active at the device when the video application was launched.illustrates the list of videosthat is displayed when the “Adults” profile is active at the device. With the “Adults” profile active, the list of videosdisplayed in user interfaceincludes videos suitable for all ages and/or children (e.g., videos-and-, rated as “All ages”) and videos that may not be suitable for children (e.g., videos-and-, rated as “17+”).illustrates the list of videosthat is displayed when the “Children” profile is active at the device. With the “Children” profile active, the list of videosdisplayed in user interfaceincludes videos suitable for all ages and/or children (e.g., videos-and-) and excludes videos that may not be suitable for children (e.g., videos-and-).

451 112 450 1714 1712 1714 1716 1716 1718 1720 1722 1722 1726 1724 1724 1726 1714 1714 1724 1726 1724 1728 1726 1724 1726 17 17 FIG.E orF 17 FIG.G Another example of an input corresponding to a request to perform one or more operations at a device is a request to play a video. For example, the user performs a gesture (e.g., a tap or double tap gesture) on touch-surface surfacewhile a focus selector (e.g., a contact detected on touchscreenor a cursor displayed on display) is located over a videoin the list of videos in user interface() to select the video. In response to detecting the gesture, video playback interfaceis displayed and playback starts (alternatively, video playback starts when the user activates a play button). Video playback interfaceincludes video frame, playback controls(e.g., play/pause button, scrubber), and volume controlfor controlling the volume level of the audio content in the video. Volume controlincludes volume slider barand thumb; the position of thumbin volume slider barindicates the current volume level. In some embodiments, if certain class-based profiles are active at the device when a videois selected for playback, a maximum volume level restriction is imposed. For example, if the “Children” profile is active when the videois selected for playback, volume thumbis restricted from moving beyond a certain level in volume slider bar. For example, in, volume thumbis restricted from moving into areain volume slider bar. This caps the maximum volume at a level below 100%, thus helping to prevent hearing damage. If the “Adults” profile is active, the maximum volume level restriction is not imposed: the user can move volume thumbalong the entire length of volume slider.

1730 1730 1732 1730 1732 1 1732 2 1732 450 17 171 FIGS.H and 17 FIG.H 17 FIG.I Another example of an input corresponding to a request to perform one or more operations at a device is a request to display a document. For example, the user makes a request to open a document (e.g., selects a document in an application or selects a hyperlink in a web browser). In response to the request, the device displays the document in a user interface (e.g., user interface). User interfaceis a user interface for any application that is capable of displaying documents or text, such as a word processing application, web browser, e-reader application, and so on. In some embodiments, the device automatically adjusts the zoom scale of the displayed document or the font size of the text in the displayed document based on the active class-based profile. For example,show documentdisplayed in user interface. If the active class-based profile is “Adults,” the zoom level is set to a default or normal level (e.g., 100%), as shown with document-in. If the active class-based profile is “Children” or a class-based profile corresponding to seniors, the zoom level is set to a higher level (e.g., 200%), as shown with document-in. With the higher zoom level, text in documentappears bigger on displayand thus easier for a child or a senior citizen to read.

1702 1706 359 1 1702 359 1 1702 1706 359 1 1706 17 FIG.A 17 FIG.B In some embodiments, when a fingerprint (e.g., fingerprintor) is detected on fingerprint sensor-, a user-specific profile is identified and selected as well as the one or more class-based profiles. For example, when fingerprintis detected on fingerprint sensor-(), the device identifies a matching fingerprint from the previously registered fingerprints on the device and selects a user-specific profile associated with the identified fingerprint matching fingerprintto be the active user-specific profile on the device, as well identifying and selecting one or more class-based profiles to be active on the device, as described above. Similarly, when fingerprintis detected on fingerprint sensor-(), the device identifies a matching fingerprint from the previously registered fingerprints on the device and selects a user-specific profile associated with the identified fingerprint matching fingerprintto be the active user-specific profile on the device, as well identifying and selecting one or more class-based profiles to be active on the device, as described above.

17 FIG.J 1734 450 1734 1734 1734 1736 1734 1734 1736 1734 1738 1 1738 6 1738 1736 In some embodiments, when the device receives an input corresponding to a request to perform one or more operations, the device performs a respective operation, in response to the request, based on the active user-specific profile. For example,illustrates web form(e.g., a web page with fillable form fields) for inputting personal information displayed on displayin response to a user input corresponding to a request to display web formin an application (e.g., a web browser). The input includes, for example, the user selecting (e.g., by tap gesture on; by mouse click while a focus selector is located over) a hyperlink to web form. Web formincludes personal information fields, such as name, one or more address-related fields (e.g., street, city, etc.), and phone number. In response to receiving the request to display web form, the device displays web formand populates fieldsin web formwith personal information-thru-based on the active user-specific profile (e.g., a name, address and phone number for “John Doc”). Personal informationused to populate fieldsare drawn from information associated with the active user-specific profile, such as contact information associated with the active user-specific profile.

17 FIG.K 1740 450 1740 1740 1740 1740 1742 1742 1744 1746 As another example,illustrates personalized web pagedisplayed on displayin response to a user input corresponding to a request to load web pagein an application (e.g., a web browser). The input includes, for example, the user selecting (e.g., by tap gesture on; by mouse click while a focus selector is located over) a hyperlink to web pageor completing a login procedure for loading web page. Web pageincludes content personalized to the user associated with the active user-specific profile. The personalized content includes, for example, shortcuts or linksto web applications or other website features, where the set of shortcutsbeing presented is personalized to the user's preferences or historical pattern of web usage; news feed, where the news categories and/or news sources are personalized to the user's preferences; and weatherpersonalized to a location associated with the user (e.g., home address, work address, current location).

1740 1742 1744 1746 1740 1740 102 370 Web pagewith personalized content,, andis loaded in accordance with a cookie associated with the active user-specific profile; the host of web pageidentifies the user to which web pageis personalized based on the cookie. The cookie is stored in, for example, the memory (e.g., memoryor) of the device.

18 18 FIGS.A-B 3 FIG. 1 FIG.A 1800 1800 300 100 1800 are flow diagrams illustrating a methodof performing operations based on a class-based profile in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display, a fingerprint sensor, and a respective input element (e.g., a touch-sensitive surface, a button or a keyboard). In some embodiments, the input element is the fingerprint sensor. In some embodiments, the input element is different from the fingerprint sensor. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

1800 As described below, the methodprovides an intuitive way to perform operations based on a class-based profile. The method reduces the cognitive burden on a user when performing operations by customizing the operations based on a class-based profile, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to perform operations based on a class-based profile faster and more efficiently conserves power and increases the time between battery charges.

1802 102 370 The device obtains (e.g., retrieving from memory or receiving from a remote computer system) a plurality of profiles (), where the plurality of profiles includes one or more user-specific profiles and one or more class-based profiles, distinct from the user-specific profiles, that each correspond to a respective class of users. The profiles are stored in, for example, the memory (e.g., memoryor) of the device or in the cloud.

1804 1702 1706 17 FIG.A 17 FIG.B The device detects () a first fingerprint on the fingerprint sensor., for example, shows the device detecting fingerprint.shows the device detecting fingerprint.

1806 The device selects () one or more class-based profiles associated with the device as active profiles that correspond to the first fingerprint without regard to authentication of the first fingerprint as a previously registered fingerprint (e.g., determining that the first fingerprint corresponds to a fingerprint of a user in the first class does not include directly comparing the first fingerprint to previously registered fingerprints), including, in accordance with a determination that the first fingerprint corresponds to a fingerprint of a user in a first class of users that are associated with a first class-based profile, selecting the first class-based profile as an active profile. In some embodiments, the first class-based profile is selected without authenticating the fingerprint as a previously registered fingerprint. For example, if no authentication is required or the fingerprint does not match any of the previously registered fingerprints, the device can still adjust the function of the device in accordance with one or more active class-based profiles that are determined based on generalized characteristics of the fingerprint and thus do not require a positive identification of a user associated with the fingerprint.

1702 1702 1706 1706 17 FIG.A 17 FIG.B For example, if fingerprintis detected (), the device selects an “Adults” class-based profile without regard to whether fingerprintmatches a previously registered fingerprint (e.g., the “Adults” class-based profile is selected based on a size of the fingerprint without needing to identify the user by matching the pattern of the fingerprint to a known fingerprint pattern associated with the user). If fingerprintis detected (), the device selects a “Children” class-based profile without regard to whether fingerprintmatches a previously registered fingerprint (e.g., the “Children” class-based profile is selected based on a size of the fingerprint without needing to identify the user by matching the pattern of the fingerprint to a known fingerprint pattern associated with the user).

1808 1702 1706 1702 In some embodiments, the first class-based profile is selected () in accordance with a detected size of the first fingerprint. For example, if the detected first fingerprint is below a predefined size threshold (e.g., average feature size, maximum diameter, etc.), then the first fingerprint is classified as a child's fingerprint and a generic child profile is selected as an active profile for the device. Thus, in some embodiments, the first fingerprint is determined to be a child fingerprint either based on identification of the first fingerprint as belonging to a user that is below a predefined or user-specified age (e.g., 12 years old) or based on a determination that the first fingerprint has fingerprint characteristics that are typical characteristics of a child's fingerprint and thus is most likely the fingerprint of a child, even if the fingerprint has not been registered as belonging to a particular child. For example, the “Adults” profile is selected for fingerprint, and the “Children” profile is selected for fingerprint, which is smaller than fingerprint.

1810 1708 1704 1708 1710 17 FIG.C 17 FIG.D In some embodiments, the first class-based profile restricts () operation of the device for members of the first class. For example, when a child-specific profile is an active profile, the device limits access to particular settings, applications, information that is inappropriate (e.g., violent or otherwise restricted) or sensitive (e.g., passwords, financial information, account settings, and/or performing destructive actions such as deleting documents and applications). For example, while the “Children” profile is active, access to settings user interfaceis restricted. Thus, if “Settings” iconis activated while the “Children” profile is active, settings user interface() is not displayed. Instead, error messageis displayed, for example as shown in.

1812 1714 1712 1714 1714 3 1714 4 17 FIG.F In some embodiments, the first class-based profile changes () functions of the device so as to make the functions more appropriate for members of the first class. For example, when a child-specific profile is an active profile, the device adapts programs for interaction with a child (e.g., using child-specific workout profiles instead of adult-specific workout profiles in a workout application or displaying children's books rather than adult books in an e-reader application). For example, while the “Children” profile is active, access to videos not suitable for children is restricted. Thus, while the “Children” profile is active, as shown in, the list of videosin user interfacelisting videosexcludes videos that may not be suitable for children (e.g., videos-and-).

1814 1816 1818 1820 1702 1706 1734 1740 17 17 FIGS.J andK In some embodiments, after detecting the first fingerprint on the fingerprint sensor (), the device determines () fingerprint features of the first fingerprint, identifies () a previously registered fingerprint matching the fingerprint features of the first fingerprint, and selects () a first user-specific profile associated with the previously registered fingerprint as an active profile (e.g., in addition to selecting the first class-based user profile as an active profile). These fingerprint analysis/matching operations are optionally performed at a central processing unit of the device or at a secured processing device associated with the fingerprint sensor. In some embodiments, the device determines whether to select any of a set of one or more user-specific profiles as an active profile and, optionally, does not select any of the plurality of profiles as an active profile if none of the user-specific profiles are determined to be associated with the first fingerprint (e.g., if none of the user-specific profiles are associated with previously registered fingerprints that match the first fingerprint). In some embodiments, if no user-specific profile is selected, a default profile is used instead. For example, after detecting fingerprintor, the device also determines if the detected fingerprint matches a previously registered fingerprint. If a previously registered fingerprint is identified for the detected fingerprint, a user-specific profile associated with the identified previously registered fingerprint is selected as active on the device, in addition to any class-based profiles selected as active (e.g., as shown in, where a web formand a webpageare loaded in accordance with user-specific information associated with a user-specific profile selected based on a detected fingerprint).

1822 451 1704 1705 1704 451 1712 451 1714 The device receives () an input with the respective input element, where the input corresponds to a request to perform one or more operations at the device. In some embodiments, the input is received while the one or more profiles associated with the device are the active profiles (e.g., after the one or more profiles have been selected as active profiles). In some embodiments, the one or more profiles are selected as the active profiles in response to receiving the input. In some embodiments, the input includes a gesture on a touch-sensitive surface of the device. In some embodiments, the input includes moving the device within range of a near-field communication device and receiving a signal from the near-field communication device. The input can be, for example, a gesture on touch-sensitive surfaceor a mouse click to activate “Settings” iconwhile a cursoris over “Settings” icon, a gesture on touch-sensitive surfaceor a mouse click to activate display of user interface, a gesture on touch-sensitive surfaceor a mouse click to select a videofor playback, and so on. In some embodiments, the input is a tap gesture on a portion of a touchscreen display that includes a selectable icon, where the fingerprint sensor is integrated into the touchscreen display and the characteristics of the fingerprint are determined based on the tap gesture. In some embodiments, the input is a press input on a trackpad (e.g., a contact with an intensity above an activation intensity threshold or a contact and activation of a physical actuator underneath the trackpad or a separate actuator button) while a cursor is over the selectable icon on a display.

1824 1708 1704 1708 1710 1714 1714 17 FIG.C 17 FIG.D 17 FIG.E 17 FIG.F In response to receiving the input with the respective input element while the first class-based profile is selected as an active profile, the device performs () a respective operation based on the request and the first class-based profile (and, optionally, based on one or more other active profiles). For example, in response to receiving the input to display settings user interface(e.g., a gesture on touch-sensitive surface to activate “Settings” icon), the device displays settings user interface() or error message() based on which class-based profile is currently active (e.g., the settings user interface is displayed if the “Adults” profile is active and the error message is displayed if the “Children” profile is active). As another example, in response to receiving the input, the device displays a full list of videos() or a limited list of videos() based on which class-based profile is currently active (e.g., the full list of videos is displayed if the “Adults” profile is active and the limited list of videos is displayed if the “Children” profile is active).

1826 1714 1714 1714 1728 1726 17 FIG.G In some embodiments, the first class-based profile is () based at least in part on an age of the user, the request includes a request to present content including audio, and the respective operation includes presenting audio adjusted for an age of the user (e.g., reducing the maximum volume for a child and/or shifting the audio frequency down for an older adult). For example, in, when playing back a video, in response to receiving the request to play back the video, while a “Children” profile is active, the device caps the maximum volume level for the audio content in the video(e.g., indicated by areain volume slider bar) to help prevent hearing damage.

1828 1732 17 FIG.I 17 FIG.H 17 FIG.I In some embodiments, the first class-based profile is () based at least in part on an age of the user, the request includes a request to present content that includes a visual component, and the respective operation includes adjusting the visual component of the media so that it is easier for the user to see the visual component (e.g., increasing display brightness or contrast, and/or increasing text font size for an older adult). For example, in, when displaying document, in response to receiving the request to display a document, while a “Children” profile or a profile corresponding to seniors is active, the device displays the document at a larger zoom scale or with a larger text font size (e.g., the document with standard sized text is displayed if the “Adults” profile is active, as shown inand the document with larger text is displayed if the “Children” profile is active, as shown in).

1830 1708 1710 17 FIG.D In some embodiments, when () the device does not have an active profile that is authorized to perform the one or more operations associated with the request, the respective operation is an error message (e.g., a message indicating that the active profile does not have permission to perform the requested operation such as accessing private information or a secure application such as a banking application). For example, when the device receives a request to display settings user interfacewhile the “Children” profile is active and “Adults” profile is not active, and thus error messageis displayed ().

1832 1714 17 FIG.G In some embodiments, in response to receiving the input with the respective input element while the first class-based profile is selected as an active profile, the device performs () the one or more operations in addition to performing the respective operation (e.g., the device performs the requested operations and customizes the operations based on the active profile, such as by changing the output of the device to be more appropriate for the approximate age of the user). For example, when the device receives a request to play a videowhile the “Children” profile is active, the video is played and the maximum volume is capped, as described above with reference to.

1834 1734 1734 1736 1738 17 FIG.J In some embodiments, the request to perform one or more operations includes () a request to display a form for inputting personal information (e.g., loading an application or webpage that includes form fields for entering personal information such as a username, address, phone number, password, or payment information associated with the first user-specific profile), and the respective operation includes automatically populating the personal information in the form based on information in the first user-specific profile. For example, in, in response to receiving a request to display web form, the device displays web formand automatically populates fieldswith personal informationbased on a user-specific profile that is currently active.

1836 1740 1740 17 FIG.K In some embodiments, the request to perform one or more operations includes () a request to load a webpage, and the respective operation includes loading the webpage in accordance with a cookie associated with the first user-specific profile. For example, in, in response to receiving a request to display web page, the device displays web pagethat is personalized to a user corresponding to a user-specific profile that is currently active based on a cookie associated with the active user-specific profile. In contrast, if a second user-specific profile is an active profile instead of the first user-specific profile being an active profile, the webpage would be loaded in accordance with a different cookie associated with the second user-specific profile.

18 18 FIGS.A-B 18 18 FIGS.A-B 1800 1800 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, gestures, and focus selectors described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, gestures, and focus selectors described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

19 FIG. 19 FIG. 1900 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

19 FIG. 1900 1902 1904 1906 1908 1902 1904 1906 1908 1910 1912 1914 1916 1918 1920 1922 As shown in, an electronic deviceincludes a display unit, a respective input element unit(e.g., a touch-sensitive surface, touchscreen display, mouse, or other input element), a fingerprint sensor unit, and a processing unitcoupled to the display unit, the respective input element unit, and the fingerprint sensor unit. In some embodiments, the processing unitincludes an obtaining unit, a detecting unit, a selecting unit, a performing unit, a determining unit, an identifying unit, and a display enabling unit.

1908 1910 1906 1912 1914 1904 1904 1916 The processing unitis configured to: obtain a plurality of profiles (e.g., with the obtaining unit), where the plurality of profiles includes one or more user-specific profiles and one or more class-based profiles, distinct from the user-specific profiles, that each correspond to a respective class of users; detect a first fingerprint on the fingerprint sensor unit(e.g., with the detecting unit); select one or more class-based profiles associated with the device as active profiles that correspond to the first fingerprint without regard to authentication of the first fingerprint as a previously registered fingerprint (e.g., with the selecting unit), including, in accordance with a determination that the first fingerprint corresponds to a fingerprint of a user in a first class of users that are associated with a first class-based profile, select the first class-based profile as an active profile; receive an input with the respective input element unit, where the input corresponds to a request to perform one or more operations at the device; and in response to receiving the input with the respective input element unitwhile the first class-based profile is selected as an active profile, perform a respective operation based on the request and the first class-based profile (e.g., with the performing unit).

In some embodiments, the first class-based profile is selected in accordance with a detected size of the first fingerprint.

In some embodiments, the class-based profile restricts operation of the device for members of the class.

In some embodiments, the class-based profile changes functions of the device so as to make the functions more appropriate for members of the class.

In some embodiments, the class-based profile is based at least in part on an age of the user, the request includes a request to present content including audio, and the respective operation includes presenting audio adjusted for an age of the user.

In some embodiments, the class-based profile is based at least in part on an age of the user, the request includes a request to present content that includes a visual component, and the respective operation includes adjusting the visual component of the media so that it is easier for the user to see the visual component.

In some embodiments, when the device does not have an active profile that is authorized to perform the one or more operations associated with the request, the respective operation is an error message.

1908 1904 1916 In some embodiments, the processing unitis configured to, in response to receiving the input with the respective input element unitwhile the first class-based profile is selected as an active profile, perform the one or more operations in addition to performing the respective operation (e.g., with the performing unit).

1908 1906 1918 1920 1914 In some embodiments, the processing unitis configured to, after detecting the first fingerprint on the fingerprint sensor unit, determine fingerprint features of the first fingerprint (e.g., with the determining unit), identify a previously registered fingerprint matching the fingerprint features of the first fingerprint (e.g., with the identifying unit), and select a first user-specific profile associated with the previously registered fingerprint as an active profile (e.g., with the selecting unit).

1922 In some embodiments, the request to perform one or more operations includes a request to enable display of a form for inputting personal information (e.g., with the display enabling unit), and the respective operation includes automatically populating the personal information in the form based on information in the first user-specific profile.

In some embodiments, the request to perform one or more operations includes a request to load a webpage, and the respective operation includes loading the webpage in accordance with a cookie associated with the first user-specific profile.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

18 18 FIGS.A-B 1 1 FIGS.A-B 19 FIG. 1 1 FIGS.A-B 1804 1806 1822 1824 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operation, selecting operation, receiving operation, and performing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices log user activity in a variety of ways. For example, some devices log activity such as internet browsing history and/or phone history. As another example, some devices log frequently used words that, despite not corresponding to words in a default dictionary, are nonetheless frequently used by the user (e.g., the word “Vladivostok” may not appear in a standard device dictionary, but will automatically be added if a user from Vladivostok, Russia types the characters “Vladivostok” into a text message a certain number of times). As yet another example, some devices begin logging activity (e.g., activity from a global positioning system, or “GPS”) when the device is determined to be lost or stolen (e.g., at the request of the user through a recovery application). However, many devices (e.g., portable multifunction devices) do not offer convenient ways to change the logging preferences. For example, when a user lends their device to another user, the device will continue to log unwanted browser history, phone history, and autocorrect inputs unless action is taken (e.g., changing settings in a setting menu) prior to the other user's use of the device. As another example, such devices will often forgo logging recovery data (e.g., GPS data) unless the user has remotely activated a logging feature (e.g., by sending a proper short message service, or “SMS”, message to the device that activates logging operations, by including, for example the text “Find my phone”).

In the embodiments described below, an improved method for changing logging preferences is achieved by detecting a fingerprint and determining if the fingerprint corresponds to a known user's profile. When it is, the user's profile is set as an active profile and a set of operations (e.g., including logging operations) is performed in accordance with the active profile. When the fingerprint does not correspond to a known user, the active profile is set to a default profile (e.g., a guest profile) and a different set of operations is performed. This method streamlines the process of changing logging preferences.

450 451 100 112 167 20 20 21 21 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 359 1 359 2 112 450 451 359 2 359 1 359 1 450 112 1 FIG.A 4 FIG.B 4 FIG.B 20 20 21 21 FIGS.A-F andA-B 20 20 FIGS.A-F 20 20 FIGS.A-F 20 20 FIGS.A-F 20 20 FIGS.A-F In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-F andA-B includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, embodiments described with reference towill be discussed with reference to device operations that are performed in response to detecting inputs described inon a touch screenand/or a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon touch screen. However, analogous operations are, optionally, performed on a device with a display, a separate touch-sensitive surfaceand a separate fingerprint sensor-with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon display. In some embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. In some embodiments, a visual representation of the focus selector is displayed (e.g., a user interface object to which the focus selector is pointing is visually distinguished from the other user interface objects).

20 20 FIGS.A-D illustrate an example of some embodiments in which a device determines that a scanned fingerprint is associated with a user's profile.

20 FIG.A 20 FIG.A 204 169 illustrates an exemplary user interface implemented on a device that changes logging preferences based on fingerprints. In, the device is locked, meaning that at least some functionality of the device is unavailable until the device is unlocked by a user (e.g., phone functionality, applications, photos, etc). In this example, the device features a home buttonthat includes an integrated fingerprint sensor.

20 FIG.B 2006 169 2004 , a fingerprintis detected on fingerprint sensor. In this example, detection of a fingerprint that is recognized by the device unlocks the device. In some embodiments, as explained below, when an unrecognized fingerprint is detected, the device unlocks to a default or guest profile. In this example, a visual cueis provided to indicate that the device is scanning a fingerprint and unlocking the device. In some embodiments, however, the device must be unlocked separately (e.g., through a gesture such as swipe-to-unlock, by entering a password, some combination thereof, or through different unlocking functionality). In some embodiments, the device can be unlocked by several users, each user having a unique user profile. When the device is unlocked by virtue of a fingerprint being recognized as belonging to a respective unique user associated with a unique user profile, the user's profile is set as an active profile, as explained below.

20 FIG.C 20 FIG.C 20 FIG.D 2006 2008 2009 As shown in the example illustrated in, the device has determined that the fingerprint is associated with a user John, unlocked the device and set John's profile as the active profile. In some embodiments, the user profile stores information (e.g., in one or more databases) as to the user's preferences, settings, browser history, phone history and the like. In some embodiments, when the device determines that scanned fingerprintbelongs to John, the device retrieves John's preferences, settings, history and the like from the databases from memory and alters the device behavior accordingly. By way of example,illustrate illustrates an input (e.g., contact) corresponding to selection of a phone application for placing phone calls. The device responds by loading the phone application, as shown in. Because John's profile is the active profile, a list of John's recent phone conversations (e.g., user interface objectsrepresenting phone conversation) is displayed to facilitate convenient call back and the ability to dial frequently dialed numbers with case.

In some embodiments, the device stores multiple user profiles. For example, when the device scans a fingerprint and determines that it is associated with a user Jane, who also has a user profile on the device, the device loads Jane's profile. By way of example, Jane's recent phone conversations are available to Jane in lieu of John's when the phone application is selected.

20 20 FIGS.A-B 20 20 FIG.E-F 20 20 FIGS.A-B 20 FIG.E 20 20 FIGS.E-F 20 FIG.E 20 FIG.D 20 FIG.F 21 21 FIGS.A-B 2006 2010 2100 In some embodiments, as illustrated inand, the device alters its behavior when a fingerprint is scanned and the device determines that the fingerprint does not correspond to a user with a user profile.have already been described with reference to the previous example, with the difference that in this example, fingerprintis not associated with a user's profile. Thus, as shown in, the device welcomes the user as a “Guest User,” indicating that a guest or default profile has been set as the active profile.offer an example of how, in some embodiments, the device alters its functionality in accordance with a default or guest profile being set as the active profile.illustrates detection of an input (e.g., a contact) corresponding selection of the phone application. In contrast to, in which an active user's recent phone conversations were shown, the device indisplays a keypad interface for the phone with no recent conversations listed. Thus, an unauthorized user is prohibited from viewing or dialing an authorized user's frequent contacts. It should be appreciated that the instant disclosure is intended to support a broad range of differences in functionality between an identified user's profile and a default user's profile. Some such optional differences in device functionality are described with reference to methoddescribed with reference to.

21 21 FIGS.A-B 3 FIG. 1 FIG.A 2100 2100 300 100 2100 are flow diagrams illustrating a methodof changing logging preferences in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a fingerprint sensor and one or more input elements, including a respective input element. In some embodiments, the device includes a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. In some embodiments, the touch-sensitive surface is the respective input element and/or the fingerprint sensor. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

2100 As described below, the methodprovides an intuitive way to change logging preferences. The method reduces the cognitive burden on a user when changing logging preferences, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to change logging preferences faster and more efficiently conserves power and increases the time between battery charges.

2102 A device with a respective input element (e.g., a touch-sensitive surface, a button or a keyboard) and a fingerprint sensor detects () a first fingerprint on the fingerprint sensor.

2104 20 20 FIGS.A-C The device determines () whether the first fingerprint is associated with a user profile associated with the device. In some embodiments, the user profile is a profile from a set of one or more user-specific profiles, generic profiles and other non-default profiles, associated with the device, that are not the default profile. In some embodiments, after detecting the first fingerprint on the fingerprint sensor and before determining whether the first fingerprint is associated with a user profile associated with the device, the device determines fingerprint features of the first fingerprint and identifies a previously registered fingerprint, if any, that matches or is associated with the fingerprint features of the first fingerprint. When the first fingerprint corresponds to a respective previously registered fingerprint associated with a non-default profile, the device determines that the fingerprint is associated with the respective non-default profile. In contrast, when the first fingerprint does not correspond to a previously registered fingerprint associated with a non-default profile, the device determines that the fingerprint is not associated with any of the non-default profiles. These fingerprint analysis/matching operations are optionally performed at a central processing unit of the device, while in other implementations the fingerprint analysis/matching operations are performed at a secured processing device associated with the fingerprint sensor. These fingerprint analysis/matching operations are optionally performed a specified time (e.g., when unlocking the device, as described with reference to). In some embodiments, fingerprint analysis and matching is performed whenever a fingerprint is detected and matched to a profile on the fingerprint (e.g., the device is nearly constantly attempting to determine the user).

2106 20 20 FIGS.C-D After determining whether the first fingerprint is associated with a user profile associated with the device, and in accordance with a determination that the first fingerprint is associated with a respective user profile associated with the device, the device sets () the respective user profile as an active profile. In some embodiments, only one user profile can be an active profile at a time. In some embodiments multiple user profiles can be active profiles at the same time. For example, a user-specific profile and one or more generic profiles are, optionally, selected as active profiles at the same time and the settings and other characteristics associated with multiple active profiles are used by the device to interpret inputs received by device. As discussed previously,illustrate an example in which the device is set to a single user's profile (e.g., a user profile belonging to John).

20 20 FIGS.E-F Also after determining whether the first fingerprint is associated with a user profile associated with the device, and in accordance with a determination that the first fingerprint is not associated with any user profile associated with the device, the device sets a default profile as an active profile. In some embodiments, when the default profile is an active profile, it is the only active profile (e.g., any other profile overrides the default profile). As discussed previously,illustrate an example in which the device is set to a default, or guest, profile.

2108 The device receives () an input with the respective input element, where the input corresponds to a request to perform a requested operation. In some embodiments, the input is received while the respective user profile or the default profile is an active profile (e.g., after the respective user profile or the default profile has been selected as the active profile). In some embodiments, the device determines whether to select the respective user profile or the default profile as the active profile in response to receiving the input.

2110 In response to receiving the input with the respective input element, in accordance with a determination that the respective user profile is the active profile, the device performs () a first set of one or more operations associated with the requested operation in accordance with logging preferences of the respective user profile. Also in response to receiving the input with the respective input element, in accordance with a determination that the default profile is the active profile, the device performs a second set of operations associated with the requested operation in accordance with logging preferences of the default profile. In some embodiments, the logging preferences of a profile correspond to instructions that indicate which activities that are performed on the device are to be logged or otherwise recorded by the device. For example, logging preferences of a first user indicate that browser history is to be tracked and application state information (e.g., open web browser tabs) is to be saved, while default logging preferences indicate that browser history is not to be tracked and that application state information is not to be saved (e.g., open web browser tabs are to be closed when the web browser ceases to be the active application).

2112 In some embodiments, performing the first set of operations includes () logging operations of a first type of operation performed on the device, and performing the second set of operations includes forgoing logging operations of the first type of operation performed on the device that would have been logged in accordance with logging preferences of the respective user profile. Thus, in some embodiments, the device does not log certain activities of temporary/unauthorized users (e.g., so as to avoid cluttering up a browsing history or recent call log of the user with activities of temporary/unauthorized users of the device).

2114 In some embodiments, the first type of operations includes () operations that are used by the device to customize operation of the device based on historical usage patterns of the device (e.g., by a primary user). For example, when the device determines that a temporary/unauthorized user is using the device, the device stops customizing one or more of: autocorrect suggestions, search history, search suggestions, browser history, username/password login information, recently used applications, recently dialed phone numbers, read/unread markers for email and/or read/unread notifications for the device until an authorized user starts to use the device again. In some embodiments, after the authorized user starts using the device again, the device starts logging the first type of operations again. Thus, in some embodiments, for a device with a single primary authorized user, the “learned” customizations of the device that are determined based on usage patterns of the authorized user are not disturbed if the user hands the device to a temporary/unauthorized user such as a friend or family member of the authorized user. Rather, while the device performs the operations requested by the temporary/unauthorized user, the device ignores those inputs for the purpose of customizing the behavior of the device in accordance with historical usage patterns.

2116 In some embodiments, performing the second set of operations includes () logging operations of a second type of operation performed on the device, and performing the first set of operations includes forgoing logging operations of the second type of operation performed on the device that would have been logged in accordance with logging preferences of the default user profile. Thus, in some embodiments, when an unauthorized user is using a device, activities of the unauthorized user are closely monitored by the device. However, when the authorized user is using the device (e.g., as determined by the fingerprints of the authorized user being detected on the fingerprint sensor), this data is not stored so as not to unnecessarily intrude upon the activities of the authorized user.

2118 In some embodiments, the second type of operation includes () operations that assist in performing forensic operations associated with a lost or stolen device (e.g., logging location information, network activity, browsing history, application use, access to private information such as bank account information, email, voicemail, etc.). Thus, in some embodiments, if the device determines that an unauthorized user is using the device, the device optionally begins tracking operations performed by the unauthorized user so that the authorized user can later be informed of the actions of the unauthorized user that may have compromised sensitive information such as financial details, personal emails, and the like.

2120 In some embodiments, the second type of operation includes () operations that assist in locating the device (e.g., logging location information identified from a GPS, WiFi or other sources). Thus, in some embodiments, if the device determines that an unauthorized user is using the device, the device optionally begins tracking the device to aid in recovery of the device.

In some embodiments, the device sends (e.g., periodically) to a remote server a log record that includes a representation of the logging operations of the second type of operation. In some embodiments, the log record is accessible (only) to an owner of the device and can be used by the owner to help identify a location of the device so as to aid in recovery of the device. Additionally, in some situations, the log records provide information that aids in determining the identity of the unauthorized user of the device, such as call history, web browsing history, location data, and/or a record of fingerprints detected on the fingerprint sensor.

21 21 FIGS.A-B 21 21 FIGS.A-B 2100 2100 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, gestures, and user interface objects, described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, gestures, and user interface objects, described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

22 FIG. 22 FIG. 2200 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

22 FIG. 2200 2202 2204 2208 2202 2204 2200 2206 2208 2210 2212 2214 2214 2216 As shown in, an electronic deviceincludes a fingerprint sensor unit; an input unit; and a processing unitcoupled to the fingerprint sensor unitand the input unit. The electronic deviceoptionally includes a display unitconfigured to enable the display of one or more user interfaces and/or one or more user interface objects. In some embodiments, the processing unitincludes a detecting unit, a fingerprint determining unit, one or more respective operation performing units. In some embodiments, one or more of the respective operation performing unitsinclude one or more logging operation units(e.g., a GPS logging unit, a browser history logging unit, etc.).

2208 2212 2208 2208 2204 2208 2208 The processing unitis configured to: detect a first fingerprint on the fingerprint sensor unitand determine whether the first fingerprint is associated with a user profile associated with the device. After determining whether the first fingerprint is associated with a user profile associated with the device, and in accordance with a determination that the first fingerprint is associated with a respective user profile associated with the device, the processing unitis configured to set the respective user profile as an active profile; and, in accordance with a determination that the first fingerprint is not associated with any user profile associated with the device, the processing unitis configured to set a default profile as an active profile. In response to receiving the input with the respective input unit, and in accordance with a determination that the respective user profile is the active profile, the processing unitis configured to perform a first set of one or more operations associated with the requested operation in accordance with logging preferences of the respective user profile; and in accordance with a determination that the default profile is the active profile, the processing unitis configured to perform a second set of operations associated with the requested operation in accordance with logging preferences of the default profile.

2216 In some embodiments, performing the first set of operations includes logging operations (e.g., performed with a logging operation unit) of a first type of operation performed on the device; and performing the second set of operations includes forgoing logging operations of the first type of operation performed on the device that would have been logged in accordance with logging preferences of the respective user profile.

In some embodiments, the first type of operations include operations that are used by the device to customize operation of the device based on historical usage patterns of the device.

2216 2216 2216 In some embodiments, performing the second set of operations includes logging operations of a second type of operation performed on the device (e.g., performed with the same logging operations unitor, alternatively, a different logging operations unitthan the logging operations unitused for logging operations of the first set of operations); and performing the first set of operations includes forgoing logging operations of the second type of operation performed on the device that would have been logged in accordance with logging preferences of the default user profile.

In some embodiments, the second type of operation includes operations that assist in performing forensic operations associated with a lost or stolen device (e.g., with a GPS logging unit).

In some embodiments, the second type of operation includes operations that assist in locating the device (e.g., with a GPS logging unit).

2208 In some embodiments, the processing unitis further configured to send to a remote server a log record that includes a representation of the logging operations of the second type of operation.

1 3 FIGS.A and The operations in the information processing methods described above are optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

21 21 FIGS.A-B 1 1 FIGS.A-B 22 FIG. 1 1 FIGS.A-B 2102 2104 2106 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operation, edit determining operation, and setting operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices are enabled to perform a restricted operation in response to a user inputting a valid authorization credential (e.g., a password or a swipe pattern). Such devices, however, rely on the restricted operation comparing a previously stored authorization credential with the inputted authorization credential in order to validate the inputted authorization credential. The device described below improves on existing methods by generating fingerprint-verification information (e.g., indicative of a “no match,” a “weak match” or a “strong match”) for a respective fingerprint in accordance with system-level fingerprint-verification criteria. In turn, the device utilizes the pre-computed fingerprint-verification information to determine whether to perform the restricted operation.

The device obtains the fingerprint-verification criteria (e.g., indicating how many verified minutia points a fingerprint needs to be assigned a respective confidence level) for controlling verification of fingerprints. Then, the device determines fingerprint-verification information for a respective input corresponding to a respective restricted operation based at least in part on: a comparison between identified fingerprint features of the respective input and fingerprint features of one or more previously registered fingerprints; and the fingerprint-verification criteria. The device only performs the respective restricted operation in accordance with a determination that the fingerprint-verification information meets operation-specific (or application-specific) authorization criteria for the respective restricted operation.

450 451 100 112 167 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 359 1 359 2 112 450 451 359 1 359 2 359 1 359 2 450 1 FIG.A 23 23 24 24 FIGS.A-G andA-D 4 FIG.B 4 FIG.B 23 23 24 24 FIGS.A-G andA-D 23 23 FIGS.A-G 23 23 FIGS.A-G In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toincludes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, cither an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screenand fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-); in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and an integrated fingerprint sensor-or a separate fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-or separate fingerprint sensor-, while displaying the user interfaces shown inon the display.

23 FIG.A 23 FIG.A 100 400 112 2310 2312 204 169 359 2 204 112 359 1 illustrates portable multifunction devicedisplaying a user interfaceon touch screenwith a plurality of applications including gaming applicationand banking application.further illustrates home buttonwith fingerprint sensor(e.g., separate fingerprint sensor-) integrated into home button. Alternatively or additionally, in some embodiments, a fingerprint sensor is integrated into at least a portion of touch screen(e.g., integrated fingerprint sensor-).

23 FIG.B 100 2302 2312 400 112 2302 2312 illustrates portable multifunction devicedetecting a user inputover banking applicationof user interfaceon touch screen. User input, for example, is a user's right index finger. Access to banking application, for example, is a restricted operation.

100 2302 2302 2312 112 112 359 1 100 2302 100 2302 2302 In some embodiments, portable multifunction deviceidentifies fingerprint features of user inputwhile user inputis over banking applicationon touch screenvia a fingerprint sensor integrated with at least a portion of touch screen(e.g., integrated fingerprint sensor-). Portable multifunction device, for example, identifies minutia points of user input, including fingerprint endings, fingerprint ridge bifurcations and short fingerprint ridges. Furthermore, portable multifunction devicedetermines fingerprint-verification information for respective user inputbased at least in part on: a comparison between the fingerprint features of the user inputand fingerprint features of one or more previously registered fingerprints; and fingerprint-verification criteria.

2302 2302 In some embodiments, the fingerprint-verification criteria specifies a plurality of classifications (or confidence values) for a detected fingerprint based on a percentage of matching minutia points in relation to one or more previously registered fingerprints. For example, user inputis classified into one of three classifications (or confidence values) according to the fingerprint-verification criteria. In this example, the fingerprint-verification information indicates that user inputis a “no match” if 30% or fewer of the detected fingerprint's minutia points match, a “weak match” if more than 30% and less than 70% of the detected fingerprint's minutia points match, or a “strong match” if more than 70% of the detected fingerprint's minutia points match. In some embodiments, a user is enabled to modify the fingerprint-verification criteria by, for example, modifying one or more of the above percentage values relating to a respective one of the three classifications.

100 2302 2302 112 100 112 359 1 2302 2312 112 100 169 359 2 204 In some embodiments, portable multifunction deviceis not configured to identify fingerprint features of user inputwhile user inputis on touch screen. In other words, portable multifunction devicedoes not have a fingerprint sensor integrated with touch screen(e.g., integrated fingerprint sensor-). In response to detecting user inputover banking application(e.g., a restricted operation) on touch screen, portable multifunction deviceprompts the user to provide a fingerprint on a dedicated fingerprint sensor(e.g., separate fingerprint sensor-) integrated with home button.

23 FIG.C 23 FIG.B 23 FIG.C 100 169 204 2302 2312 112 2304 400 112 2312 illustrates portable multifunction deviceprompting the user to provide a fingerprint on dedicated fingerprint sensorintegrated with home button, in response to detecting inputover banking application(e.g., a restricted operation) on touch screenin.illustrates text boxdisplayed on user interfaceof touch screenindicating that fingerprint identification is required for access to banking application.

23 FIG.D 23 FIG.C 100 2306 169 204 2306 169 2306 2306 illustrates portable multifunction devicedetecting a user inputover fingerprint sensorintegrated with home button, in response to the prompt in. User input, for example, is a user's right index finger. Fingerprint sensoridentifies fingerprint features of user inputand determines fingerprint-verification information for the respective input based at least in part on: a comparison between the fingerprint features of respective inputand fingerprint features of one or more previously registered fingerprints; and fingerprint-verification criteria.

23 FIG.E 23 FIG.B 23 FIG.D 23 FIG.B 23 FIG.D 2308 400 112 2312 2302 2306 2312 2312 2312 2302 2306 2310 2310 2312 2312 illustrates text boxdisplayed on user interfaceof touch screenindicating that the device is forgoing performing a respective restricted operation (e.g., denying access to banking application), in response to detecting user inputin, or user inputin, and in accordance with a determination that the respective fingerprint-verification information does not meet respective authorization criteria for the respective restricted operation (e.g., operation-specific criteria set by banking application). For example, banking application's authorization criteria requires a “strong match” to access banking application(e.g., a secured application), and the fingerprint-verification information corresponding to user inputin, or user inputin, indicates a “weak match.” In contrast, gaming application's authorization criteria requires only a “weak match” to access gaming application(e.g., an unsecured application). In this example, access to banking applicationis denied because the respective fingerprint-verification information does not meet banking application's authorization criteria.

23 FIG.F 23 FIG.B 23 FIG.D 23 FIG.B 23 FIG.D 2312 2302 2306 2312 2312 2302 2306 2312 2312 400 112 2312 illustrates performing a respective restricted operation (e.g., granting access to and displaying banking application) in response to detecting user inputin, or user inputin, and in accordance with a determination that the respective fingerprint-verification information meets respective authorization criteria for the respective restricted operation. For example, banking application's authorization criteria requires a “strong match” to access banking application, and the fingerprint-verification information corresponding to user inputin, or user inputin, indicates a “strong match.” In this example, access to banking applicationis granted and banking applicationis displayed on user interfaceof touch screenbecause the respective fingerprint-verification information meets banking application's authorization criteria.

23 FIG.G 2320 2320 100 100 2320 100 2322 100 2322 2324 359 1 359 2 2326 2328 2324 2323 2326 2325 2327 illustrates a schematic diagram of a processfor performing a respective restricted operation based on fingerprint-verification information in accordance with some embodiments. In some embodiments, processis performed at a portable multifunction deviceor a component of portable multifunction device. In some embodiments, at least a portion of processis performed at a component of portable multifunction device. For example, a fingerprint moduleis a component of portable multifunction device, and fingerprint moduleincludes one or more fingerprint sensors(e.g., including an integrated fingerprint sensor-and/or a separate fingerprint sensor-), a fingerprint memoryand fingerprint processing logiccoupled to one or more fingerprint sensorsvia lineand fingerprint memoryvia linesand.

2324 2326 2326 2328 2326 2325 2324 2323 2326 2327 2328 2337 2338 2322 One or more fingerprint sensorsare configured to identify fingerprint features of a respective input. Fingerprint memoryis configured to store fingerprint features of one or more previously registered fingerprints. In some embodiments, fingerprint memoryis further configured to also store fingerprint-verification criteria. Fingerprint processing logicis configured to determine fingerprint-verification information based at least in part on: fingerprint-verification criteria obtained from fingerprint memoryvia line; and a comparison of the fingerprint features of the respective input obtained from one or more fingerprint sensorsvia lineand fingerprint features of one or more previously registered fingerprints obtained from fingerprint memoryvia line. Fingerprint processing logicis further configured to provide the fingerprint-verification information via lineto respective restricted operation module. In some embodiments, fingerprint moduleconsists of secured dedicated computing hardware that has additional security features so as to enhance security of the fingerprint-verification information.

2332 2328 2332 2328 2336 In some embodiments, matching fingerprint identification moduleof fingerprint processing logicidentifies a matching fingerprint in accordance with the fingerprint-verification criteria. In some embodiments, best matching fingerprint identification moduleof fingerprint processing logicidentifies a best matching fingerprint in accordance with the fingerprint-verification criteria. Furthermore, in some embodiments, confidence assignment moduleassigns a match confidence value to the best matching fingerprint.

2338 2337 2328 2340 2342 2328 2344 2328 Respective restricted operation module(e.g., a banking application) obtains fingerprint-verification information via linefrom fingerprint processing logicand determines via determination modulewhether the fingerprint-verification information meets respective authorization criteria for the respective restricted operation. Performance moduleperforms the restricted operation (e.g., grants access to the banking application) when that the fingerprint-verification information obtained from fingerprint processing logicmeets the respective authorization criteria for the restricted operation. Forgoing moduleforgoes performance of the restricted operation (e.g., denies access to the banking application) when the fingerprint-verification information obtained from fingerprint processing logicdoes not meet the respective authorization criteria for the restricted operation.

24 24 FIGS.A-D 3 FIG. 1 FIG.A 2400 2400 300 100 2400 are flow diagrams illustrating a methodof performing a respective restricted operation based on fingerprint-verification information in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a fingerprint sensor. In some embodiments, the display is a touch screen display and the fingerprint sensor is on the display. In some embodiments, the display is separate from the fingerprint sensor. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

2400 As described below, the methodprovides an intuitive way to perform a respective restricted operation based on fingerprint-verification information. The method reduces the cognitive burden on a user when performing a respective restricted operation based on fingerprint-verification information, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to perform a respective restricted operation based on fingerprint-verification information faster and more efficiently conserves power and increases the time between battery charges.

2402 2322 2328 2326 2325 23 FIG.G The device obtains () (e.g., retrieves from memory or receives from a remote computer system) or accesses (e.g., in memory or in one or more instructions of fingerprint module) fingerprint-verification criteria for controlling (e.g., determining or generating) verification of fingerprints. For example, the fingerprint-verification criteria sets a number of minutia points of a fingerprint that need to be verified to determine that a detected fingerprint is a previously registered fingerprint and/or the fingerprint-verification criteria sets how many verified minutia points of a fingerprint need to be verified for the fingerprint to be assigned different confidence levels indicative of a confidence of the match between the detected fingerprint and the previously registered fingerprint., for example, shows fingerprint processing logicobtaining fingerprint-verification criteria from fingerprint memoryvia linefor determining fingerprint-verification information.

2404 In some embodiments, the fingerprint-verification criteria includes () a set of default criteria; and obtaining the fingerprint-verification criteria includes: receiving a criteria-adjustment input from a user indicating a modification to the fingerprint-verification criteria; and in response to receiving the criteria-adjustment input, replacing one or more default criteria with corresponding modified criteria in the fingerprint-verification criteria. For example, while in a user-adjustable settings mode, the device receives a user input specifying the system-level verification criteria. In some embodiments, changing the fingerprint-verification criteria will change which fingerprints are matched with which inputs and/or the confidence level of the matches. Thus, in some embodiments, an input that would have matched with a first fingerprint at a first confidence level using the default fingerprint-verification criteria would instead match the first fingerprint at a second confidence level (different from the first confidence level) using the modified fingerprint-verification criteria.

In some circumstances, the fingerprint-verification criteria are adjusted so that a closer match between the respective input and a previously registered fingerprint is required to identify the respective input as matching the previously registered fingerprint or reach a particular confidence level that the respective input matches the previously registered fingerprint. For example, the user increases the security of the device by decreasing the likelihood of false positive matches in exchange for a corresponding increase in the likelihood of false negative matches. In some circumstances, the fingerprint-verification criteria are adjusted (e.g., by reducing a required confidence level for fingerprint matching) so that a less close match between the respective input and a previously registered fingerprint will be accepted to identify the respective input as matching the previously registered fingerprint. For example, the user decreases the security of the device by increasing in the likelihood of false positive matches in exchange for a corresponding decrease in the likelihood of false negative matches.

2406 2408 169 204 100 2306 23 FIG.D For each of a plurality of respective inputs that correspond () to respective restricted operations, the device identifies () fingerprint features of the respective input., for example, shows fingerprint sensorintegrated with home buttonof portable multifunction deviceidentifying fingerprint features of user input.

2406 2410 2412 2414 100 2306 169 204 2306 23 FIG.D 23 FIG.D 23 FIG.D Furthermore, for each of a plurality of respective inputs that correspond () to respective restricted operations, the device determines () fingerprint-verification information for the respective input based at least in part on: a comparison () between the fingerprint features of the respective input and fingerprint features of one or more previously registered fingerprints; and the fingerprint-verification criteria (). As one example, portable multifunction devicedetects a fingerprint (e.g., user inputin) on the fingerprint sensor (e.g., fingerprint sensorintegrated with home buttonin), identifies features of the fingerprint, and compares the features of the fingerprint with features of previously registered fingerprints. In many circumstances a match between features of a detected fingerprint (e.g., user inputin) and a previously registered fingerprint will not be a perfect match (e.g., the detected fingerprint matches 5 out of 10 minutia points), and thus it is ambiguous whether or not the detected fingerprint matches the previously registered fingerprint. In order to resolve this ambiguity, the device optionally applies fingerprint-verification criteria to the results of the comparison between the features of the detected fingerprint and the features of the previously registered fingerprint. For example, the fingerprint-verification criteria specify that a detected fingerprint that matches 3 or fewer minutia points (30% or fewer) is classified as “not matching,” while a detected fingerprint that matches between 4-6 minutia points (more than 30% and less than 70%) is classified as a “weak match,” and a detected fingerprint that matches 7 or more minutia points (70% or more) is classified as a “strong match.”

2416 In some embodiments, the device uses () the same fingerprint-verification criteria to process fingerprints for multiple different (e.g., sequential) respective inputs. For example, the fingerprint-verification criteria is system-level criteria or operation-independent criteria and is applied to a comparison between the detected fingerprint and the previously registered fingerprints to generate fingerprint-verification information without regard to the operation for which the fingerprint-verification information will be used.

2418 2332 23 FIG.G In some embodiments, the device determines () the fingerprint verification information by: identifying minutia points in the respective input; comparing the minutia points identified in the respective input with minutia points in a plurality of candidate fingerprints; and selecting a respective candidate fingerprint with a highest number of matching minutia points as the matching fingerprint. In some embodiments, the minutia points for a fingerprint include one or more of: fingerprint ridge endings, fingerprint ridge bifurcations, and short fingerprint ridges (sometimes called dots). A fingerprint ridge ending is a point at which a fingerprint ridge terminates. A fingerprint ridge bifurcation is a point at which a single fingerprint ridge splits into two fingerprint ridges. A short fingerprint ridge is a ridge that is significantly shorter than an average ridge length of a fingerprint ridge in the fingerprint., for example, shows matching fingerprint identification moduleconfigured to select a respective candidate fingerprint with a highest number of matching minutia points as the matching fingerprint.

2420 2334 23 FIG.G In some embodiments, the device determines () the fingerprint verification information by: identifying minutia points in the respective input; comparing the minutia points identified in the respective input with minutia points in a plurality of candidate fingerprints; and selecting a respective candidate fingerprint with a highest number of matching minutia points as the best matching fingerprint., for example, shows best matching fingerprint identification moduleconfigured to select a respective candidate fingerprint with a highest number of matching minutia points as the best matching fingerprint.

2422 2424 2426 2428 2336 23 FIG.G In some embodiments, the device assigns () a first confidence value to the best matching fingerprint, in accordance with a determination that a first number of minutia points in the respective input match corresponding minutia points in the best matching fingerprint. The device further assigns () a second confidence value to the best matching fingerprint, in accordance with a determination that a second number of minutia points in the respective input match corresponding minutia points in the best matching fingerprint. The second number of minutia points is greater () than the first number of minutia points, and the second confidence value indicates () a closer match between the respective input and the best matching fingerprint than is indicated by the first confidence value., for example, shows confidence value assignment moduleconfigured to assign a respective confidence value to the best matching fingerprint. In some embodiments, when a second number of minutia points in the respective input match corresponding minutia points in the best matching fingerprint, only the second confidence value is assigned to the best matching fingerprint.

2430 2432 2332 23 FIG.G In some embodiments, the fingerprint-verification information includes () information identifying () a matching fingerprint determined in accordance with the fingerprint-verification criteria., for example, shows matching fingerprint identification moduleconfigured to include information identifying a matching fingerprint in the fingerprint-verification information.

2430 2434 2334 23 FIG.G In some embodiments, the fingerprint-verification information includes () information identifying () a best matching fingerprint of a plurality of previously registered fingerprints, where the best matching fingerprint is identified in accordance with the fingerprint-verification criteria; and a fingerprint match confidence value indicative of a confidence of a match between the first input and the best matching fingerprint, where the fingerprint match confidence value is determined in accordance with the fingerprint-verification criteria., for example, shows best matching fingerprint identification moduleconfigured to include information identifying a best matching fingerprint and a match confidence value in the fingerprint-verification information.

2436 2438 2312 2302 112 2306 169 2302 2306 2312 23 FIG.E 23 FIG.B 23 FIG.D 23 FIG.B 23 FIG.D In response to detecting () the respective input, the device forgoes () performance of the respective restricted operation, in accordance with a determination that the fingerprint-verification information does not meet the respective authorization criteria for the restricted operation (e.g., operation-specific authorization criteria). For example, the respective authorization criteria indicate which previously registered fingerprints are authorized to perform the respective restricted operation and optionally indicate a minimum confidence level that is to be met in order for the respective restricted operation to be performed., for example, shows portable multifunction device denying access to banking application, in response to detecting user inputon touch screenin, or user inputon fingerprint sensorin, and in accordance with a determination that the fingerprint-verification information (e.g., generated in response to the detection of user inputin, or user inputin) does not meet the respective authorization criteria for banking application.

2436 2440 100 2312 2302 112 2306 169 2302 2306 2312 23 FIG.F 23 FIG.B 23 FIG.D 23 FIG.B 23 FIG.D In response to detecting () the respective input, the device performs () the respective restricted operation, in accordance with a determination that the fingerprint-verification information meets respective authorization criteria for the respective restricted operation., for example, shows portable multifunction devicegranting access to and displaying banking application(e.g., the respective restricted operation), in response to detecting user inputon touch screenin, or user inputon fingerprint sensorin, and in accordance with a determination that the fingerprint-verification information (e.g., generated in response to the detection of user inputin, or user inputin) meets the respective authorization criteria for banking application.

2310 2310 2312 2312 For example, after the detected fingerprint has been classified in accordance with the fingerprint-verification criteria, this fingerprint-verification information (e.g., “no match,” “weak match” or “strong match”) is passed to individual applications to handle in accordance with application-specific or operation-specific authorization criteria. For example in some circumstances, a non-secure application (e.g., an application that relies on the fingerprint-verification information only for user identification), such as gaming application, uses authorization criteria that only requires a weak match, and thus gaming applicationwill accept the detected fingerprint as a first fingerprint if the detected fingerprint is classified as either a “weak match” or a “strong match” for the first fingerprint (e.g., in accordance with the fingerprint-verification criteria). In contrast, in some circumstances, a secure application (e.g., an application that relies on the fingerprint-verification information to determine whether or not to permit access to sensitive information), such as banking application, uses authorization criteria that requires a strong match, and thus banking applicationwill only accept the detected fingerprint as a first fingerprint if the detected fingerprint is classified as a “strong match” for the first fingerprint (e.g., in accordance with the fingerprint-verification criteria).

2310 2312 100 100 100 In the example above, gaming applicationand banking applicationboth rely on the pre-computed fingerprint-verification information (e.g., “no match,” “weak match” or “strong match”) generated in accordance with the fingerprint-verification criteria rather than comparing the detected fingerprint and the previously registered fingerprints directly. As a preliminary matter, performing the comparison process once and providing the pre-computed fingerprint-verification information to multiple applications or processes can substantially decrease the computing resources used by device, thereby increasing the energy efficiency, battery life, and responsiveness of device. Additionally, having the different applications and processes use the pre-computed fingerprint-verification information enables deviceor the user to easily adjust the security of multiple fingerprint sensitive operations or applications quickly and efficiently by adjusting the fingerprint-verification criteria.

100 2312 2310 100 For example, devicecan adjust the fingerprint-verification criteria by increasing the number (or percentage) of minutia point matches needed for a “strong match” of a detected fingerprint with a previously registered fingerprint (e.g., without changing the authorization information for one or more operations or applications), so as to strengthen security of secure applications (e.g., banking application) without affecting the case of use of non-secure applications (e.g., gaming application). Similarly, devicecan adjust the fingerprint-verification criteria by decreasing the number (or percentage) of minutia point matches needed for a “strong match” and/or a “weak match” (e.g., without changing the authorization information for one or more operations or applications), so as to decrease the incidence of false negative matches (e.g., the user may choose to accept weaker security if the device repeatedly fails to correctly identify the user's fingerprints).

2442 2444 100 2442 2446 In some embodiments, the operation-specific criteria are met () when the matching fingerprint is () a fingerprint associated with a user authorized to perform the restricted operation. For example, the matching fingerprint is associated with the user profile of the user who logged into (or unlocked) portable multifunction device. In some embodiments, the operation-specific criteria are met () when the best matching fingerprint is () associated with a user authorized to perform the respective restricted operation, and the confidence value is above a confidence threshold associated with the respective restricted operation.

2448 2312 2310 2310 2312 In some embodiments, the plurality of restricted operations include (): a first restricted operation that is associated with a first confidence threshold; and a second restricted operation that is associated with a second confidence threshold different from the first confidence threshold. Thus, in some embodiments, where the first input includes fingerprint features that match the best matched fingerprint with a confidence value that is above the first confidence threshold and below the second confidence threshold, the first input would enable the user to perform the first restricted operation but would not enable the user to perform the second restricted operation. For example, a confidence threshold for identifying a user for making electronic payments via banking applicationis higher than the confidence threshold for identifying a user for gaming application. Thus a quick, low resolution scan of a user's fingerprint would suffice for identifying the user for gaming application, while a more careful, high resolution scan of the user's fingerprint is, optionally, required to authorize an electronic payment via banking applicationfor the user.

2450 169 204 2306 169 100 2306 204 204 100 2312 2312 23 FIG.D 23 FIG.D 23 FIG.D 23 FIG.F In some embodiments, the fingerprint sensor is integrated () into a physical button; in response to detecting the fingerprint on the fingerprint sensor, the device determines the fingerprint-verification information; and in response to detecting activation of the button, the device performs the respective restricted operation. In some embodiments, the physical button is an intensity-sensitive region of the housing of the device that is determined to have been activated by the device when the intensity detected on the button is increased over an activation intensity threshold. In some embodiments, the physical button is coupled to an electronic switch where movement of the physical button along a first axis (e.g., up and down or left to right) closes the switch and activates the physical button (sometimes resulting in an audible or tactile “click”)., for example, shows fingerprint sensorintegrated with home button. In response to detecting inputon fingerprint sensorin, for example, portable multifunction devicedetermines fingerprint-verification information corresponding to input. And further, in response to detecting activation of home button(e.g., downward movement of buttonin), portable multifunction devicegrants access to banking applicationand displays banking applicationin.

24 24 FIGS.A-D 24 24 FIGS.A-D 2400 2400 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the inputs fingerprints, contacts described above with reference to methodoptionally have one or more of the characteristics of the inputs fingerprints, contacts described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

25 FIG. 25 FIG. 2500 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

25 FIG. 2500 2502 2504 2506 2502 2504 2506 2508 2510 2512 2514 2516 2518 2520 As shown in, an electronic deviceincludes a display unitconfigured to display one or more user interface objects, a fingerprint sensor unit; and a processing unitcoupled to the display unitand the fingerprint sensor unit. In some embodiments, the processing unitincludes an obtaining unit, an identifying unit, a determining unit, an authorizing unit, a performing unit, an assigning unitand a replacing unit.

2506 2508 2506 2510 2512 2506 2514 2516 2514 The processing unitis configured to obtain (e.g., with obtaining unit) or otherwise access fingerprint-verification criteria for controlling verification of fingerprints. For each of a plurality of respective inputs that correspond to respective restricted operations, the processing unitis further configured to identify (e.g., with identifying unit) fingerprint features of the respective input and determine (e.g., with determining unit) fingerprint-verification information for the respective input based at least in part on: a comparison between the fingerprint features of the respective input and fingerprint features of one or more previously registered fingerprints; and the fingerprint-verification criteria. In response to detecting the respective input, the processing unitis configured to: in accordance with a determination (e.g., with authorizing unit) that the fingerprint-verification information meets respective authorization criteria for the respective restricted operation, perform (e.g., with performing unit) the respective restricted operation; and in accordance with a determination (e.g., with authorizing unit) that the fingerprint-verification information does not meet the respective authorization criteria for the restricted operation, forgo performance of the respective restricted operation.

In some embodiments, the same fingerprint-verification criteria is used to process fingerprints for multiple different respective inputs.

In some embodiments, the fingerprint-verification information includes information identifying a matching fingerprint determined in accordance with the fingerprint-verification criteria.

In some embodiments, the operation-specific authorization criteria are met when the matching fingerprint is a fingerprint associated with a user authorized to perform the restricted operation.

2510 2512 2512 In some embodiments, the processing unit is configured to determine the fingerprint verification information by: identifying (e.g., with identifying unit) minutia points in the respective input; comparing (e.g., with determining unit) the minutia points identified in the respective input with minutia points in a plurality of candidate fingerprints; and selecting (e.g., with determining unit) a respective candidate fingerprint with a highest number of matching minutia points as the matching fingerprint.

In some embodiments, the fingerprint-verification information includes: information identifying a best matching fingerprint of a plurality of previously registered fingerprints, where the best matching fingerprint is identified in accordance with the fingerprint-verification criteria; and a fingerprint match confidence value indicative of a confidence of a match between the first input and the best matching fingerprint, where the fingerprint match confidence value is determined in accordance with the fingerprint-verification criteria.

In some embodiments, the operation-specific authorization criteria are met when: the best matching fingerprint is associated with a user authorized to perform the respective restricted operation; and the confidence value is above a confidence threshold associated with the respective restricted operation.

In some embodiments, the plurality of restricted operations include: a first restricted operation that is associated with a first confidence threshold; and a second restricted operation that is associated with a second confidence threshold different from the first confidence threshold.

2510 2512 2512 In some embodiments, the processing unit is configured to determine the fingerprint verification information by: identifying (e.g., with identifying unit) minutia points in the respective input; comparing (e.g., with determining unit) the minutia points identified in the respective input with minutia points in a plurality of candidate fingerprints; and selecting (e.g., with determining unit) a respective candidate fingerprint with a highest number of matching minutia points as the best matching fingerprint.

2518 2518 In some embodiments, the processing unit is configured to: in accordance with a determination that a first number of minutia points in the respective input match corresponding minutia points in the best matching fingerprint, assign (e.g., with assigning unit) a first confidence value to the best matching fingerprint; and in accordance with a determination that a second number of minutia points in the respective input match corresponding minutia points in the best matching fingerprint, assign (e.g., with assigning unit) a second confidence value to the best matching fingerprint, where: the second number of minutia points is greater than the first number of minutia points; and the second confidence value indicates a closer match between the respective input and the best matching fingerprint than is indicated by the first confidence value.

2506 2508 2508 2520 In some embodiments, the fingerprint-verification criteria includes a set of default criteria; and the processing unitis configured to obtain (e.g., with obtaining unit) the fingerprint-verification criteria by: receiving (e.g., with obtaining unit) a criteria-adjustment input from a user indicating a modification to the fingerprint-verification criteria; and in response to receiving the criteria-adjustment input, replacing (e.g., with replacing unit) one or more default criteria with corresponding modified criteria in the fingerprint-verification criteria.

2504 2506 2512 2504 2506 2516 In some embodiments, the fingerprint sensor unitis integrated into a physical button; the processing unitis configured to determine (e.g., with determining unit) the fingerprint-verification information in response to detecting the fingerprint on the fingerprint sensor unit; and the processing unitis configured to perform (e.g., with the performing unit) the respective restricted operation in response to detecting activation of the button.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

24 24 FIGS.A-D 1 1 FIGS.A-B 25 FIG. 1 1 FIGS.A-B 2402 2408 2410 2440 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, obtaining operation, identifying operation, determining operations, and performing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices allow users to interact with their environment through interactive maps displayed on a display. For example, such electronic devices (e.g., portable multifunction devices) often allow users to indicate their presence at a particular business or well-known landmark on social media sites. In such situations, such electronic devices often display a map on a touch screen display and a user can interact with the map by, for example, tapping on the location of the business or well-known landmark. As another example, such electronic devices often include cameras, and it is desirable to provide users with a convenient way to associate identity and location metadata with photographs taken using the device. For example, in some embodiments, a user can indicate the location of a photograph by tapping on a corresponding location on a map displayed on a touch screen display and by further indicating the identity author and/or the subjects of the photograph through separate inputs. Existing methods of associating identity and location information with, for example, a social media post or as metadata associated with a photograph typically require separate inputs indicating each of identity and location.

In the embodiments described below, an improved method for associating identity and location information is achieved by detecting an input that is associated with both a displayed location on a map (e.g., a location that corresponds to a geographic location) and with a fingerprint detected on a fingerprint sensor. The method identifies a user that corresponds to the fingerprint and performs an operation associated with the location and the user, such as posting an indication of the user's presence at the location to social media or adding metadata indicating the location of a photograph and that the user is a subject and/or author of the photograph.

450 451 100 112 167 26 26 27 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 169 359 1 359 2 112 450 451 359 2 359 1 359 1 450 112 1 FIG.A 4 FIG.B 4 FIG.B 26 26 27 FIGS.A-G and 26 26 FIGS.A-G 26 26 FIGS.A-G 26 26 FIGS.A-G 26 26 FIGS.A-G In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-G andincludes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, embodiments described with reference towill be discussed with reference to device operations that are performed in response to detecting inputs described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon touch screen. However, analogous operations are, optionally, performed on a device with a display, a separate touch-sensitive surfaceand a separate fingerprint sensor-with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon display. In some embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor.

26 FIG.A 2601 2602 2604 2601 illustrates a device displaying a user interfacethat includes display of a map. In this example, the user interface is displayed as part of an application that assists users in navigating to local business and landmarks. For example, when the user searches on the term “Gala Coffee” in search bar, the device returns several results and displays their locations as pin-dropson user interface.

26 FIG.B 27 FIG. 2606 112 2608 169 2606 2606 2606 112 2606 169 illustrates detection of a contacton touch screenand detection of a fingerprinton a separate fingerprint sensor. Contactis an example of an activation input that triggers an operation of the device that is associated with both the location of contacton the map (e.g., a physical location corresponding to the displayed location on the map at which contactis received) as well as the identified fingerprint. In some embodiments (e.g., when touch screenis configured to identify fingerprints), a single contact acts as both an activation input and as an input for which a fingerprint is identified (e.g., a fingerprint is detected corresponding to contact). In some embodiments, a fingerprint is detected on separate fingerprint sensorand another type of activation input, such as a speech input, is detected by the device and associated with a particular location on the map, as described in greater detail with respect to.

26 26 FIGS.C-G 26 26 FIGS.C-G 2606 illustrate examples of various operations that the device performs according to some embodiments. As noted above, these operations are associated with both the location of contacton the map as well as the identified fingerprint. It should be understood that the present disclosure is intended to support a broad range of operations that are associated with a location and identity, and that the operations shown inare merely exemplary.

26 FIG.C 2604 2 2605 2605 illustrates an example in which the operation includes indicating the presence of the user (e.g., the user whose fingerprint was identified) at a particular “Gala Coffee” store. In this example, the user has selected a particular pin-drop corresponding to the particular Gala Coffee shop-. In some embodiments, the device displays a confirmation messageasking the user to confirm that they want to “Check in” at the particular “Gala Coffee” and identifying the address of the location of the check in. For example, activating the “CHECK IN” affordance in confirmation messagewill cause the device to post the user's presence on a social media website.

26 FIG.D 26 FIG.C 2606 2604 2 2610 2612 2610 2614 illustrates an alternate (or in some circumstances, additional) example of an operation that the device performs in response to contact, in accordance with some embodiments. As was the case in, the user has selected a pin-drop corresponding to the “Gala Coffee”-store. However, in this example, the device responds by prompting the user to submit a payment amount through payment prompt. The user can enter an amount using keypadwithin promptand then select “CONFIRM” buttonto submit the payment. In some embodiments, a user can link different accounts to different fingers. For example, a right thumb fingerprint will cause payment to be submitted through the user's debit accord whereas a left index fingerprint will cause payment to be submitted through the user's credit account.

26 26 FIGS.E-G 26 FIG.E 26 FIG.E 26 FIG.E 2616 2615 2615 2615 2616 2615 illustrate an alternate example of a device operation performed in response to a contact, in accordance with some embodiments.illustrates display of a photographof the Golden Gate Bridge taken from a location in San Francisco, California. In some circumstances, photographis a photograph that was previously taken by the device (e.g., when the device is equipped with a camera phone). In some circumstances, photographis a photograph that was previously taken by a different device and transmitted to the device (e.g., over a network, by email, downloaded from the internet, etc.). As shown in, the device optionally displays a prompt asking the user if the user would like to add metadata (in this example, location and author data) to the photo.also illustrates detection of a contactindicating in the affirmative that the user would like to association location and author metadata with photograph.

26 FIG.F 26 FIG.E 26 FIG.E 26 FIG.G 26 FIG.G 2616 2618 2615 2620 2622 169 2618 2620 2622 continues the example illustrated in. Upon selection of the “YES” affordance invia contact, the device displays a mapfor associating metadata including location information (e.g., geotagging) with photograph. The device detects a contactat a location on the map that has special significance for the photograph (e.g., where the photograph was taken). The device also detects a fingerprinton fingerprint sensorcorresponding to a user and identifies a user to whom it belongs. As shown in, the device associates data associated with both the location on mapof contact(in this case, the coordinate location of the Golden Gate Bridge, San Francisco, CA), as well as the identity of the user to whom fingerprintbelongs (e.g., Joseph Strauss, as shown in).

27 FIG. 3 FIG. 1 FIG.A 2700 2700 300 100 2700 is a flow diagram illustrating a methodof associating identity and location information in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

2700 As described below, methodprovides an intuitive way to associate identity and location information. The method reduces the cognitive burden on a user when associating identity and location information, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to associate identity and location information faster and more efficiently conserves power and increases the time between battery charges.

2702 2702 169 26 FIG.A The device displays () a map on a display of an electronic device with a fingerprint sensor (). For example,illustrates a device with a fingerprint sensordisplaying a map.

2704 2606 26 FIG.B The device detects () an activation input, the activation input being associated with a displayed location on the map that corresponds to a respective geographic location and a fingerprint detected on the fingerprint sensor. For example, the activation input is performed while a focus selector is at the displayed location on the map in order to associate the activation input with the displayed location on the map and the corresponding geographic location (e.g., contactin). In some embodiments, the activation input is associated with the fingerprint detected on the fingerprint sensor in accordance with a determination that the activation input is performed while the fingerprint is detected on the fingerprint sensor.

26 FIG.A 2604 1 2608 2604 1 In some embodiments, the activation input is a speech input. For example, while displaying the map shown inand detecting the fingerprint on the fingerprint sensor, the user will say “I'm at Gala Coffee on 1st and Cedar,” which the device will interpret as an activation input for the particular Gala Coffee (represented by pin-drop-) at the intersection of Fast 1st Street and Fast Cedar Street. In some embodiments, the activation input is a speech input that corresponds to a request for the device to use location determination functionality of the device to determine the user's location. For example, while (or after, or before) fingerprintis detected, the user will say “Use my current location,” and the device will determine the user's current location (e.g., using global positioning system software and/or hardware). Alternatively, the user will say “I'm at Gala Coffee,” or “Check in at Gala Coffee” and the device will recognize the input as a request to check in the identified user at the nearest Gala Coffee (e.g., the Gala Coffee represented by pin-drop-when the device determines that user is located at the intersection of East 1st Street and East Cedar Street).

2706 26 26 FIGS.A-G In some embodiments, the activation input is detected () on a touch-sensitive surface different from the fingerprint sensor (e.g., the activation input is detected on a portion of a touchpad or touch screen display that does not include an integrated fingerprint sensor, the activation input occurring concurrently or substantially concurrently with a fingerprint being placed on the integrated fingerprint sensor, as shown in).

2708 In some embodiments, the activation input is detected () on the fingerprint sensor (e.g., the activation input is detected on a portion of a touchpad or touch screen display that includes an integrated fingerprint sensor, and the integrated fingerprint sensor is used to identify the user that corresponds to the fingerprint).

2710 2712 2714 In response to detecting the activation input (), the device identifies () a user that corresponds to the fingerprint (e.g., identifying a user profile that includes a previously registered fingerprint that matches the fingerprint detected on the fingerprint sensor). The device then performs () an operation associated with the respective geographic location and the user.

2716 26 26 FIGS.A-C In some embodiments, the operation associated with the respective geographic location and the user includes () generating a message indicating the (virtual or physical) presence of the user at the respective geographic location (e.g., the user can “check in” to a geographic location such as a restaurant, store, park, or other place of business by selecting a location on the map that corresponds to the place of business while simultaneously having a fingerprint on a fingerprint sensor, as shown in). In some embodiments, the generated message indicates that the user is currently physically present at the respective geographic location. In some embodiments, the generated message indicates that the user was previously physically present or will be physically present at the respective geographic location in the future. In some embodiments, the message indicating the presence of the user indicates an actual physical presence of the user at the location (e.g., the user checks in at a restaurant while the user is eating at the restaurant to indicate that the user is eating at the restaurant). In some embodiments, the message indicating the presence of the user indicates a virtual presence of the user (e.g., the user checks in at a restaurant to request a reservation at the restaurant or indicate that the user has made a reservation at the restaurant, or the user checks in at a location where a meeting is occurring even though the user is participating in the meeting remotely via phone or video conference).

2718 26 FIG.D In some embodiments, the operation associated with the geographic location and the user includes () authorizing a payment from the user to a business at the respective geographic location (e.g., a step in authorizing a payment to a store is to select a location on the map that corresponds to the location of the store while a fingerprint is on a fingerprint sensor, as shown in).

2720 26 26 FIGS.E-G In some embodiments, the operation associated with the geographic location and the user includes () adding metadata to media, the metadata indicating that the media is associated with the respective geographic location and the user (e.g., geotagging a photo taken by the user or of the user by adding metadata to the photo that indicates both a location where the photo was taken and that the user took the photo or appears in the photo, as shown in).

27 FIG. 27 FIG. 2700 2700 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g. those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, inputs, maps, and focus selectors described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, inputs, maps, and focus selectors described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

28 FIG. 28 FIG. 2800 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

28 FIG. 2800 2802 2806 2808 2802 2806 2804 2806 1 2804 2804 1 2806 2 2808 2810 2812 2814 As shown in, an electronic deviceincludes a display unitconfigured to display a user interface including a map, a fingerprint sensor unit; and a processing unitcoupled to the display unitand the fingerprint sensor unit. In some embodiments, the device includes a touch-sensitive surface unitwith an incorporated fingerprint sensor unit-. In such embodiments, the processing unit is coupled to the touch-sensitive surface unittogether with the fingerprint sensor unit-. In some embodiments, the device includes a separate fingerprint sensor unit-. In some embodiments, the processing unitincludes a detecting unit, an identifying unit, and a performing unit.

2808 2806 2808 The processing unitis configured to detect an activation input, the activation input being associated with a displayed location on the map that corresponds to a respective geographic location and a fingerprint detected on the fingerprint sensor unit. In response to detecting the activation input: the processing unitis configured to identify a user that corresponds to the fingerprint and perform an operation associated with the respective geographic location and the user.

2804 2804 In some embodiments, the activation input is detected on a touch-sensitive surface unitdifferent from the fingerprint sensor unit.

2804 In some embodiments, the activation input is detected on the fingerprint sensor unit.

In some embodiments, the operation associated with the respective geographic location and the user includes generating a message indicating the presence of the user at the respective geographic location, as discussed in more detail above.

In some embodiments, the operation associated with the geographic location and the user includes authorizing a payment from the user to a business at the respective geographic location, as discussed in more detail above.

In some embodiments, the operation associated with the geographic location and the user includes adding metadata to media, the metadata indicating that the media is associated with the respective geographic location and the user, as discussed in more detail above.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

27 FIG. 1 1 FIGS.A-B 28 FIG. 1 1 FIGS.A-B 2702 2704 2712 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, display operation, detecting operation, and identifying operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have graphical user interfaces that allow users to perform a variety of functions and operations associated with the respective user interfaces (e.g., data storage options, text editing applications, drawing applications, and the like). Furthermore, many electronic devices are shared between two or more users. While interacting with such shared devices, a user frequently desires to create and define a personal workspace or work area with custom-defined functions and operations (e.g., a custom list of applications, personal data storage space, custom data formatting options such as user-specified text fonts, line widths, drawing options and the like) based on the user's own preferences.

Some approaches for users to create their own personal workspace on a shared device include creation of separate user accounts (e.g., with separate user names and passwords to associate users to their respective accounts). The users can then log into their respective accounts using their respective user names and passwords. These approaches are limited in a manner that only one user would be able to log into his or her respective account at any given time on the shared device, thereby making it difficult for multiple users to concurrently access their accounts on the same shared device. Additionally, these approaches require navigation through a hierarchy of menus and settings to switch between workspaces which can be confusing and time-consuming for users.

The disclosed embodiments provide a convenient and efficient method of partitioning a shared workspace on a multifunction device by enabling a user to define a area on a display of the multifunction device by using one or more contacts to define the perimeter or boundary of the area. Based at least on a fingerprint detected in the one or more contacts used by the user to define the area, the device associates the area with the respective user and associates the area with the user's personal preferences and settings. As a result, one or more users can quickly and efficiently define respective workspaces or work areas on the same physical device and have their custom settings associated with the respective workspaces based on fingerprint identification of the users.

450 451 100 112 167 29 29 30 30 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 450 451 359 2 359 1 359 1 450 112 169 359 1 359 2 112 112 1 FIG.A 4 FIG.B 4 FIG.B 29 29 30 30 FIGS.A-S andA-B 29 29 FIGS.A-S 29 29 FIGS.A-S 29 29 FIGS.A-S 29 29 FIGS.A-S In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-S andA-B includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to display, a separate touch-sensitive surfaceand a separate fingerprint sensor-, however analogous operations are, optionally, performed on a device with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display. Additionally, analogous operations are, optionally, performed on a device with a touch screenin response to detecting the contacts described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon the touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a displayed cursor.

29 FIG.A 424 446 2900 100 2900 2900 112 illustrates a first user interface (e.g., an application launch user interface or a home screen interface), with user selectable icons (e.g., user-selectable icons-corresponding to a plurality of launchable applications), that is displayed on a displayof a portable multi-function device. In some embodiments, displayis a touch-sensitive display with an array of touch sensors that are responsive to touch. In some embodiments, the array of touch sensors is in a plane coincident (e.g., collocated or co-planar) with a plane that defines the display elements forming display. A combined displayand touch-sensitive surface is sometimes referred to herein as a touch screen. In some embodiments, the device includes a touch sensitive surface (e.g., in a plane coincident, collocated or coplanar with the display) with an array of touch sensors that are responsive to touch and the touch sensitive surface includes a fingerprint sensing capability (e.g., the touch sensitive surface includes an array of fingerprint sensors capable of detecting fingerprints with high spatial precision). Thus, in some embodiments, the device is capable of detecting one or more fingerprints on the touch sensitive surface and optionally associating the one or more fingerprints with one or more corresponding users of the device.

29 29 FIGS.A-B 29 29 FIGS.A-B 29 29 FIGS.A-B 112 112 illustrate detecting a first set of one or more inputs (e.g., including one or more contacts with one or more corresponding fingerprints) on the touch screenthat define a first area (e.g., delineate a first perimeter that bounds a first area) on the touch screen.further illustrate determining that a first contact in the one or more contacts includes a first fingerprint associated with a first user. Accordingly,illustrate that the device associates the first area with the first user (e.g., as a workspace for the first user with custom work settings specific to the first user).

29 FIG.A 29 FIG.B 29 FIG.B 2902 2902 2902 2902 112 100 2902 2902 2902 2902 2902 2902 2902 2902 112 a b c d a b c d a b c d As shown in, the device detects a first set of one or more inputs including contacts with corresponding fingerprints-,-,-, and-) on the touch screenof device. The first set of inputs (e.g., including corresponding fingerprints-,-,-, and-) define a (e.g., substantially rectangular) perimeter that forms the border or boundary of a first (e.g., substantially rectangular) area on the touch screen surface of the device. As shown in, the device determines that one or more of the fingerprints-,-,-, and-is associated with a first user (e.g., Jim). Accordingly, the device associates (e.g., designates) the first area of the touch screenwith the first user (e.g., based on a comparison of the detected fingerprints with fingerprint features of fingerprints of the user that determines that the detected fingerprints match fingerprints of the user). For example, the first area is a workspace that is designated to Jim and includes Jim's preferred (e.g., custom-defined) applications (e.g., and corresponding application launch icons). As shown in, a user has the improved convenience of carving out, outlining or designating his or her personal workspace (e.g., from a shared workspace), on a multifunction device, with his or her custom preferred settings, simply by using one or more inputs (e.g., with one or more contacts having one or more corresponding fingerprints) to define the boundary or perimeter of an area for the personal workspace.

29 FIG.C 29 29 FIGS.C-D 112 100 2906 2906 2906 2906 112 a b c d Along similar lines,illustrates a second user interface (e.g., a notepad or text editing application user interface) with a list of notes (e.g., most recently created or edited notes by one or more users), in a common workspace, that is displayed on touch screenof a portable multi-function device.illustrate detecting a first set of one or more inputs (e.g., including one or more fingerprints-,-,-, and-) associated with a first user (e.g., Mark) on the touch screen, where the first set of one or more inputs defines a first area.

29 FIG.D 29 FIG.D 29 29 FIGS.E-F 29 29 FIGS.E-F 2908 2910 112 2908 2910 As shown in, the first set of one or more inputs is associated with (e.g., defines the boundary or perimeter of) a first area (e.g., rectangular text editing workspace) associated with the first user (e.g., with Mark) based at least on determining that the first set of inputs includes one or more fingerprints corresponding to the first user. As shown in, text entered by the first user in the first area (e.g., the new note by Mark entered by way of a second set of one or more inputs) is associated with a first operation (e.g., text entered in the first area is displayed and formatted in accordance with custom settings such as font, text size, formatting and the like) based on the preferences of the first user (e.g., Mark). As shown in, the device detects a third set of one or more inputs (e.g., contact) on the touch screenoutside the first area (e.g., outside the rectangular text editing workspace). If the device detected inputs corresponding to creating a new note that were not associated with the area of the display, then the new note would be generated using default preferences (e.g., different font, text size and formatting preferences). As shown inin accordance with a determination that the third set of one or more inputs (e.g., contact) is outside the first area, the device performs a second operation (e.g., minimizes the new note by Mark to redisplay the list of recently created or edited notes).

Thus, in some embodiments, after carving out, outlining, defining, or designating his or her personal workspace (e.g., from a shared workspace), on a multifunction device, the user has the improved convenience of performing a first operation (e.g., in accordance with or associated with his or her custom preferred settings) simply by placing one or more contacts inside the first area (e.g., corresponding to his or her personal workspace); and of performing a second operation (e.g., dismissing the personalized workspace or generating a new note in accordance with default preferences) by placing one or more contacts outside the first area.

29 FIG.G 29 FIG.H 29 FIG.I 112 100 2912 2912 2912 2912 112 2913 a b c d As yet another example,illustrates a third user interface (e.g., a drawing application user interface), with a common shared (e.g., between one or more users) workspace for creating drawings, that is displayed on touch screenof portable multi-function device.illustrates detecting a first set of one or more inputs (e.g., including one or more fingerprints-,-,-, and-) associated with a first user (e.g., Mark) on the touch screen. As shown in, the first set of one or more inputs defines a first area (e.g., rectangular drawing workspace) associated with the first user (e.g., with Mark) based at least on determining that the first set of inputs includes one or more fingerprints corresponding to the first user (e.g., corresponding to Mark).

29 29 FIGS.J-K 29 29 FIGS.K-L 29 FIG.L 2914 112 2913 2914 2913 2914 2916 112 2913 further illustrate detecting a second set of one or more inputs (e.g., a contact) on the touch screenwithin the first area (e.g., within Mark's rectangular drawing workspace) and attributing the contact to the first user. For example, upon detecting contactwithin Mark's drawing workspace, the device performs a first operation in accordance with custom settings associated with the first user (e.g., the device fills in the shape and outlines the shape drawn in the first area, by movement of contact, based on Mark's preferences or custom settings such as color preferences, boundary settings and the like). On the other hand, as shown inthe device detects a third set of one or more inputs (e.g., a contact) on the touch screenoutside the first area (e.g., outside Mark's rectangular drawing workspace) and performs a second operation (e.g., in accordance with or associated with default settings, such as default color or fill and boundary settings that are distinct from Mark's custom settings as shown in).

29 29 FIGS.M-Q 29 29 FIGS.M-Q 29 FIG.Q 29 FIG.Q 2918 112 2919 2918 2919 As shown in, while the first area is associated with the first user, a fourth set of one or more inputs (e.g., including one or more instances of a moving fingerprint) associated with a second user (e.g., Jane) is detected on the touch screen. As shown in, the fourth set of one or more inputs defines a second area (e.g., rectangular drawing workspace) associated with the second user (e.g., with Jane) based at least on determining that the fourth set of inputs includes one or more fingerprints (e.g., fingerprint) corresponding to the second user (e.g., corresponding to Jane). As shown in, the second area (e.g., rectangular drawing workspace) is proximate to the vertical right edge of the display (e.g., with reference to the orientation of the device shown in). Accordingly, the second area (including one or more user interface objects in the second area) is oriented using the vertical right edge of the display as the bottom of the second area display.

29 29 FIGS.R-S 29 FIG.S 2920 112 2919 2920 2919 2919 2919 As shown in, the device detects a second set of one or more inputs (e.g., a contact) on the touch screenwithin the second area (e.g., within Jane's rectangular drawing workspace) and attributes the contactto the second user (e.g., to Jane). For example, upon detecting contactwithin Jane's drawing workspace, the device performs a second operation in accordance with custom settings associated with the second user (e.g., the device fills in the shape and outlines the shape drawn in the second area by movement of contactbased on Jane's preferences or custom settings such as color, boundary settings and the like). As shown in, Jane's preferences specify a thinner line width and a lighter fill for objects than the line width and fill specified by Mark's preferences.

2913 2919 2913 2919 Thus, in some embodiments, after a first user defines his or her personal workspace (e.g., from a shared workspace) on a multifunction device, a second user can concurrently define his or her own workspace (e.g., distinct from the workspace of the first user). As a result, two or more users can simultaneously share a workspace on a multifunction device by partitioning it into their respective work areas using a perimeter defined by their respective finger contacts; the device then associates the respective work areas of the individual users with the respective users based on their respective fingerprints. In some embodiments, the separate workspaces correspond to different documents (e.g., a drawing document for Mark and a separate drawing document for Jane, so that when areasandare dismissed, the objects drawn by Mark and Jane will be saved in separate documents associated with the respective user who drew the object). In some embodiments, the separate workspaces correspond to different workspaces in the same document (e.g., Mark and Jane are making contributions to different portions of the same document, so that when areasandare dismissed, the objects drawn by Mark and Jane will continue to be shown in the same document).

30 30 FIGS.A-B 3 FIG. 1 FIG.A 3000 3000 300 100 3000 are flow diagrams illustrating a methodof assigning associating areas of a touch-sensitive surface with one or more users in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

3000 As described below, the methodprovides an intuitive way to associating areas of a touch-sensitive surface with one or more users. The method reduces the cognitive burden on a user when selecting a workspace, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to select or self-designate a workspace faster and more efficiently conserves power and increases the time between battery charges.

3002 3004 2904 2908 2913 29 29 FIGS.M-P 29 29 29 FIGS.A,C, andH 29 FIG.B 29 FIG.D 5 FIG.I The device detects () a first set of one or more inputs on the touch-sensitive surface that define a first area of the touch-sensitive surface. For example, the device detects a user outlining an area of the touch-sensitive surface with a finger (e.g., as explained with reference to) or detects a user placing 2, 3 or 4 fingers on the display simultaneously to identify corners of a workspace (e.g., as explained with reference to). In some embodiments, the first area of the touch-sensitive surface is () an area of a touchscreen display (e.g., first area,; first area,; first area,).

3006 3008 2902 2902 2902 2902 2906 2906 2906 2906 2912 2912 2912 2912 3010 2904 2908 2913 a b c d a b c d a b c d 29 29 FIGS.A-B 29 29 FIGS.C-D 29 291 FIGS.H- 29 FIG.B 29 FIG.D 29 FIG.I 29 FIG.B 29 FIG.D 29 FIG.K After detecting () the first set of one or more inputs (e.g., in response to detecting the first set of one or more inputs): the device determines () that the first set of one or more inputs includes a first contact that includes a first fingerprint associated with a first user (e.g., fingerprints-,-,-, and-associated with Jim,; fingerprints-,-,-, and-associated with Mark,; fingerprints-,-,-, and-associated with Mark,). The device associates () the first area of the touch-sensitive surface with the first user based at least in part on the determination that the first set of one or more inputs includes the first contact that includes the first fingerprint associated with the first user (e.g., first areaassociated with Jim,; first areaassociated with Mark,; first areaassociated with Mark,). Thus, in some embodiments, a user is able to select a portion of a shared workspace on the touch-sensitive surface as the user's workspace by tracing an outline of an area in the shared workspace using one or more contacts that include fingerprints associated with the user. The user is then able to use this individual workspace to perform user-specific operations that are associated with the user (e.g., use custom applications as explained with reference to; use a custom fonts and letter sizes as explained with reference to; use a custom line width and boundary setting and shape color or fill as explained with reference to).

3012 2904 2908 2913 29 FIG.B 29 FIG.D 29 FIG.I In some embodiments, after associating the first area of the touch-sensitive surface with the first user, the device displays () a visual indication of extent of the first area. For example, the device displays a border around the first area (e.g., a border around first areaassociated with Jim,; or around first areaassociated with Mark,; or around first areaassociated with Mark,).

3014 3016 3034 3016 3018 In some embodiments, while the first area of the touch-sensitive surface is associated () with the first user, the device performs one on more steps described with reference to operations-. The device detects () a second set of one or more inputs within the first area. The device attributes () the second set of one or more inputs to the first user. For example, the device performs one or more operations in response to detecting the second set of one or more inputs within the first area, stores data generated in accordance with the second set of one or more inputs in a storage location associated with the first user or otherwise treats the second set of one or more inputs as though they are inputs from the first user. In some embodiments, the second set of one or more inputs are attributed to the first user based on the location of the inputs in the first area without requiring other indications that the second set of one or more inputs correspond to the user. For example, the device does not identify fingerprints of contacts in the second set of one or more inputs. Instead, gestures that occur in the first area are assumed to be inputs by the first user.

3020 3022 2914 2913 2910 2908 2916 2913 29 FIG.D 29 29 FIGS.J-K 29 FIG.E 29 29 FIGS.K-L In some embodiments, the device detects () a third set of one or more inputs. In response to detecting () the third set of one or more inputs: in accordance with a determination that the third set of one or more inputs are detected within the first area, the device performs a first operation (e.g., the device performs a text editing operation in accordance with the first user's font preferences as explained with reference to; or upon detecting contactwithin the first area, the device uses the first user's custom line width and shape fill for a drawing, as explained with reference to); and in accordance with a determination that the third set of one or more inputs are detected outside of the first area, the device performs a second operation different from the first operation (e.g., upon detecting contactoutside the first area, the device minimizes the new note created by the first user and redisplays a list of recently created or edited notes, explained with reference to; upon detecting contactoutside the first area, the device uses a default line width and shape fill for a drawing, as explained with reference to). For example, inputs within the first area are translated into visual elements in accordance with user preferences of the first user (e.g., text font preference, text color preference, line color/width preference, custom dictionary or autocorrect options, or custom application toolbar settings associated with the first user), while inputs outside of the first area are translated into visual elements in accordance with default user preferences (e.g., default text font preference, default text color preference, default line color/width preference, default dictionary or autocorrect options, or default application toolbar settings).

3024 2918 2919 3026 2918 3028 2919 2918 29 FIG.Q 29 29 FIGS.M-Q 29 FIG.Q In some embodiments, the device detects () a fourth set of one or more inputs on the touch-sensitive surface that define a second area of the touch-sensitive surface (e.g., movement of contact including fingerprintthat defines second area,). For example, the device detects a second user outlining an area of the touch-sensitive surface (e.g., as explained with reference to) with a finger or placing 2, 3 or 4 fingers on the display simultaneously to identify corners of a workspace for the second user. The device determines () that the fourth set of one or more inputs includes a second contact that includes a second fingerprint associated with a second user (e.g., fingerprintassociated with Jane,). The device associates () the second area of the touch-sensitive surface with the second user based at least in part on the determination that the fourth set of one or more inputs includes the second contact that includes the second fingerprint associated with the second user (e.g., the device associated second areawith Jane based at least on determining that fingerprintis associated with Jane). In some embodiments, the second area of the touch-sensitive surface is distinct from (e.g., non-overlapping with) the first area of the touch-sensitive surface.

3030 3032 2914 2913 2920 2919 29 29 FIGS.J-K 29 29 FIGS.R-S In some embodiments, while the first area of the touch-sensitive surface is associated with the first user and the second area of the touch-sensitive surface is associated with the second user, the device detects () a fifth set of one or more inputs. In response to detecting the fifth set of one or more inputs: in accordance with a determination () that the fifth set of one or more inputs are detected within the first area, the device performs a first operation (e.g., upon detecting contactwithin the first area, the device uses custom settings for line width or shape fill based on Mark's preferences, as explained with reference to); and in accordance with a determination that the fifth set of one or more inputs are detected within the second area, the device performs a second operation different from the first operation (e.g., upon detecting contactwithin the second area, the device uses custom settings for line width or shape fill based on Jane's preferences, as explained with reference to). For example, inputs within the first area are translated into visual elements in accordance with user preferences of the first user (e.g., text markup color, text font preference, text color preference, line color/width preference, custom dictionary or autocorrect options, and/or custom application toolbar settings associated with the first user), while inputs within the second area are translated into visual elements in accordance with user preferences of the second user (e.g., text markup color, text font preference, text color preference, line color/width preference, custom dictionary or autocorrect options, and/or custom application toolbar settings associated with the second user).

2913 2919 29 29 FIGS.G-S 29 29 FIGS.I-L 29 29 FIGS.G-S 29 29 FIGS.Q-S In some embodiments, the first area is proximate to a first edge of the display (e.g., first areais proximate to the horizontal lower edge of the display with reference to the orientation of the device shown in); one or more first user interface objects displayed in the first area are oriented using the first edge as the bottom of the display (e.g., Mark's drawing workspace is oriented using the horizontal lower edge as the bottom, as explained with reference to); the second area is proximate to a second edge of the display, different from the first edge of the display (e.g., second areais proximate to the vertical right edge of the display with reference to the orientation of the device shown in); and one or more second user interface objects displayed in the second area are oriented using the second edge as the bottom of the display (e.g., Jane's drawing workspace is oriented using the vertical right edge as the bottom, as explained with reference to). For example, the device orients text/graphical objects in the first area so that “bottoms” of objects/text are towards an edge of the display closest to the first area. Similarly, the device orients text/graphical objects in the second area so that “bottoms” of objects/text are towards an edge of the display closest to the second area.

30 30 FIGS.A-B 30 30 FIGS.A-B 3000 3000 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, user interface objects, inputs, visual indications described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, user interface objects, inputs, visual indications described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

31 FIG. 31 FIG. 3100 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

31 FIG. 3100 3102 3104 3104 3106 3108 3102 3104 3106 3108 3110 3112 3114 3116 3118 3120 As shown in, an electronic deviceincludes a display unitconfigured to display one or more user interface objects, a touch-sensitive surface unitconfigured to receive a contact on the touch-sensitive surface unit, a fingerprint sensor unit; and a processing unitcoupled to the display unit, the touch-sensitive surface unit, and the fingerprint sensor unit. In some embodiments, the processing unitincludes a detecting unit, a determining unit, an associating unit, a display enabling unit, an attributing unit, and a performing unit.

3108 3104 3104 3110 3112 3104 3114 The processing unitis configured to: detect a first set of one or more inputs on the touch-sensitive surface unitthat define a first area of the touch-sensitive surface unit(e.g., with the detecting unit); and after detecting the first set of one or more inputs: determine that the first set of one or more inputs includes a first contact that includes a first fingerprint associated with a first user (e.g., with the determining unit); and associate the first area of the touch-sensitive surface unitwith the first user based at least in part on the determination that the first set of one or more inputs includes the first contact that includes the first fingerprint associated with the first user (e.g., with the associating unit).

3104 In some embodiments, the first area of the touch-sensitive surface unitis an area of a touchscreen display.

3108 3104 3116 In some embodiments, the processing unitis configured to, after associating the first area of the touch-sensitive surface unitwith the first user, enable display of a visual indication of extent of the first area (e.g., with the display enabling unit).

3108 3104 3110 3118 In some embodiments, the processing unitis configured to, while the first area of the touch-sensitive surface unitis associated with the first user: detect a second set of one or more inputs within the first area (e.g., with the detecting unit); and attribute the second set of one or more inputs to the first user (e.g., with the attributing unit).

3108 3104 3110 3120 3120 In some embodiments, the processing unitis configured to: while the first area of the touch-sensitive surface unitis associated with the first user, detect a third set of one or more inputs (e.g., with the detecting unit); and in response to detecting the third set of one or more inputs: in accordance with a determination that the third set of one or more inputs are detected within the first area, perform a first operation (e.g., with the performing unit); and in accordance with a determination that the third set of one or more inputs are detected outside of the first area, perform a second operation different from the first operation (e.g., with the performing unit).

3108 3104 3104 3110 3112 3104 3114 In some embodiments, the processing unitis configured to: while the first area of the touch-sensitive surface unitis associated with the first user, detect a fourth set of one or more inputs on the touch-sensitive surface unit that define a second area of the touch-sensitive surface unit(e.g., with the detecting unit); determine that the fourth set of one or more inputs includes a second contact that includes a second fingerprint associated with a second user (e.g., with the determining unit); and associate the second area of the touch-sensitive surface unitwith the second user based at least in part on the determination that the fourth set of one or more inputs includes the second contact that includes the second fingerprint associated with the second user (e.g., with the associating unit).

3108 3104 3104 3110 3120 3120 In some embodiments, the processing unitis configured to: while the first area of the touch-sensitive surface unitis associated with the first user and the second area of the touch-sensitive surface unitis associated with the second user, detect a fifth set of one or more inputs (e.g., with the detecting unit); and in response to detecting the fifth set of one or more inputs: in accordance with a determination that the fifth set of one or more inputs are detected within the first area, perform a first operation (e.g., with the performing unit); and in accordance with a determination that the fifth set of one or more inputs are detected within the second area, perform a second operation different from the first operation (e.g., with the performing unit).

3102 3102 3102 3102 3102 In some embodiments, the first area is proximate to a first edge of the display unit; one or more first user interface objects displayed in the first area are oriented using the first edge as the bottom of the display unit; the second area is proximate to a second edge of the display unit, different from the first edge of the display unit; and one or more second user interface objects displayed in the second area are oriented using the second edge as the bottom of the display unit.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

30 30 FIGS.A-B 1 1 FIGS.A-B 31 FIG. 1 1 FIGS.A-B 3002 3008 3010 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operation, determining operation, and associating operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Effectively performing tasks such as typing on an electronic device and playing a piano or electric keyboard require a user to use proper fingering techniques. For example, a technique known as “touch typing” was developed to improve a user's speed and accuracy when typing on a keyboard (e.g., a QWERTY keyboard associated with an electronic device or typewriter). This technique involves a user lightly placing eight fingers (excluding their right and left thumbs) along the middle row of letters on a QWERTY keyboard (e.g., the user's left pinky finger, ring finger, middle finger, and index finger are places on the “A”, “S”, “D”, and “F” keys, respectively, and the user's right index finger, middle finger, ring finger, and pinky finger are placed on the “J”, “K”, “L”, and “;” keys, respectively). Using this set-up as a base hand position, each key of the keyboard is assigned a finger that should be used to depress the key, causing the letter associated with the depressed key to by typed on a connected device (e.g., an electronic device such as a computer, or typewriter). When proficient at touch typing, a user can rely on muscle memory, rather than visual identification, to type a string of letters, increasing their efficiency. Conventional methods for training a user on proper typing technique include instructions for a user to depress a particular key with a corresponding finger and/or perform a series of practice exercises. However, while the computer can provide feedback as to the accuracy of the letters that are typed, computer cannot provide the user with feedback as to whether they have used the correct finger to depress the respective keys on the keyboard.

Likewise, playing a musical keyboard with high proficiency requires proper fingering technique, both to produce consistent sound quality (e.g., by depressing the keys with proper technique) and to achieve adequate speed and accuracy (e.g., by simultaneously depressing a combination of keys in a chord or serially depressing a series of keys in a musical passage). Conventional methods for training a musician to play a musical keyboard (e.g., a piano) include one-on-one (e.g., “private”) lessons with an instructor, who is able to demonstrate proper technique and provide feedback to the learning musician. However, private lessons can be prohibitively expensive and typically must be scheduled well in advance. Alternatively, programs exist that enable a user to connect a keyboard to an electronic device or allow use of an electronic device with a touch-sensitive surface as a proxy keyboard. Similar to programs used to learn typing on a computer, these programs are able to provide feedback on the accuracy of the notes being played, but not the user's fingering technique.

In embodiments described below, improved methods and user interfaces for training proper fingering techniques (e.g., for typing or playing a musical keyboard) are achieved at an electronic device by detecting finger contacts and analyzing the fingerprints associated with the finger contacts. In this fashion, the electronic device can determine which fingers, and in some embodiments which portion of the fingertips, were used to make the contacts. Advantageously, unlike conventional electronic methods for training typing and musical keyboard playing, the methods and user interfaces described below provide feedback indicating whether proper fingering techniques are being used (e.g., whether the correct fingers are being used for typing, whether the user is contacting a musical keyboard at the correct elevation and direction, and whether appropriate combinations of fingers are being used to play musical chords and passages). Furthermore, the methods and user interfaces described below can be used to self-teach, eliminating the costs and scheduling restrictions associated with taking private music lessons.

450 451 100 112 167 32 32 33 33 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 359 1 112 359 1 359 1 450 1 FIG.A 4 FIG.B 4 FIG.B 32 32 33 33 FIGS.A-S andA-C 32 32 FIGS.A-S 32 32 FIGS.A-S In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-S andA-C includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference toto a touch screenwith an integrated fingerprint sensor-(e.g., a touch screenwith a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges), however analogous operations are, optionally, performed on a device with a separate fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display.

32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 32 FIGS.C-K andM 100 112 100 169 3200 112 3200 3201 3202 1 3202 24 3240 112 3240 3222 3232 1 3232 41 3210 3208 111 112 3201 illustrate multifunction devicehaving a touch screenwith a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, allowing them to be used as fingerprint sensors. In some embodiments, multifunction devicealternatively, or in addition, has a separate fingerprint sensor.illustrate user interfacedisplayed on touch screen, user interfaceincluding musical keyboardhaving a plurality of activatable user interface objects (e.g., keys-to-).illustrate user interfacedisplayed on touch screen, user interfaceincluding QWERTY character entry keyboardhaving a plurality of activatable user interface objects (e.g., keys-to-).include audio frequency metershowing the one or more sound frequencies of audio outputgenerated at speakerin response to an audio output generating event (e.g., detection of a qualifying contact at a position on touch screencorresponding to the display of a key on musical keyboard). These audio frequency meters are typically not part of the displayed user interface, but are provided to aid in the interpretation of the figures.

32 32 FIGS.A-P 100 112 112 3208 illustrate various embodiments where multifunction devicedetects a fingerprint pattern of a contact at a position on touch screencorresponding to display of an activatable music key in a musical keyboard, and based upon one or more characteristics of the fingerprint pattern, provides feedback to the user if the fingering used to make the contact on touch screenis not a preferred fingering and plays a note (e.g., generated audio output) if the fingering used to make the contact is a preferred fingering.

32 32 FIGS.A-G 32 FIG.A 100 3204 112 3202 5 3201 100 3204 3204 3202 5 For example,illustrate various embodiments in which multifunction devicedetects a fingerprint pattern of contactat a position on touch screencorresponding to display of key-(e.g., the “E3” key) on musical keyboard, as illustrated in. Multifunction devicedetermines whether the fingerprint pattern of contactcorresponds to a preferred fingering, the preferred fingering including at least a criterion that the finger corresponding to the fingerprint pattern of contactused to activate “E3” key-is the user's left middle finger.

32 32 32 FIGS.B,D-F 32 32 32 FIGS.B,D-F 3205 3202 5 100 3205 3205 100 illustrate various embodiments in which the fingering used to make contactis not a preferred fingering based upon at least a criterion that the “E3” key-is contacted by the user's left middle finger. Multifunction deviceanalyzes a fingerprint pattern of contact(e.g., including at least a determination of the identity of the finger corresponding to the fingerprint pattern) and determines that contactinwas made by the user's left ring (LR) finger. Because the user's left ring finger does not match the predetermined preferred finger (e.g., the user's left middle finger), multifunction deviceprovides feedback to the user indicating that a preferred fingering was not used.

32 32 FIGS.B andD 3202 5 3206 112 In some embodiments, the feedback indicating that the user has used the wrong finger (e.g., their left ring finger instead of their left middle finger) includes visual feedback. For example, as illustrated in, when “E3” key-is contacted with the user's left ring (LR) finger, large “X”is displayed on touch screen.

32 FIG.E 3202 5 100 3208 111 3210 In some embodiments, the feedback indicating that the user has used the wrong finger (e.g., their left ring finger instead of their left middle finger) includes audible feedback. For example, as illustrated in, when “E3” key-is contacted with the user's left ring (LR) finger, multifunction devicegenerates audio outputat speakerhaving a frequencyof 161.00 Hz, which is out of tune with a true “E3” tone, which has a frequency of 164.81 Hz. Particularly when played in a sequence of notes (e.g., as part of a musical passage), or in a chord, the user perceives that the note is “off,” indicating that they have used a non-preferred fingering.

32 FIG.F 3202 5 100 3212 167 In some embodiments, the feedback indicating that the user has used the wrong finger (e.g., their left ring finger instead of their left middle finger) includes tactile feedback. For example, as illustrated in, when “E3” key-is contacted with the user's left ring (LR) finger, multifunction devicegenerates tactile output(e.g., using tactile output generator).

32 32 FIGS.C andG 32 32 FIGS.C andG 32 FIG.C 32 FIG.G 3207 3202 5 100 3207 3207 100 3208 111 100 100 3218 112 illustrate various embodiments in which the fingering used to make contactis a preferred fingering based upon at least a criterion that the “E3” key-is contacted by the user's left middle (LM) finger. Multifunction deviceanalyzes a fingerprint pattern of contact(e.g., including at least a determination of the identity of the finger corresponding to the fingerprint pattern) and determines that contactinwas made by the user's left middle (LM) finger. Because the user's left middle finger is the predetermined preferred finger, multifunction devicegenerates audio outputat speakerhaving a frequency of 164.81 Hz, corresponding to the true frequency of an “E3” tone. In, in accordance with some embodiments, multifunction devicedoes not provide feedback indicating that the preferred fingering was used. In, in accordance with some embodiments, multifunction deviceprovides feedback indicating that the preferred fingering was used by displaying staron touch screen.

32 32 FIGS.H-J 32 32 FIGS.H-J 3214 3217 3216 3202 1 3202 5 3202 8 3201 illustrate various embodiments in which the preferred fingering to play a C-major triad includes a user's left pinky finger, left middle finger, and left thumb. For example, the preferred fingering criteria includes criterion that the fingerprint patterns of contacts,, and, used to activate “C” key-, “E3” key-, and “G” key-on musical keyboardin, respectively, correspond to the preferred fingering.

32 32 FIGS.H-I 32 32 FIGS.H-I 32 32 FIGS.H-I 32 FIG.H 32 FIG.I 3214 3217 3216 3217 3202 5 3201 100 3206 112 100 3206 112 3208 111 3208 illustrate various embodiments in which the fingering used to make contacts,, andinis not the preferred fingering, because the fingerprint pattern of contactin, used to activate “E3” key-on musical keyboard, corresponds to the user's left ring (LR) finger, rather than the user's left middle finger. In, in accordance with some embodiments, multifunction deviceprovides visual feedback indicating that a non-preferred fingering was used by displaying large “X”on touch screen. In, in accordance with some embodiments, multifunction deviceprovides visual and audio feedback indicating that a non-preferred fingering was used by displaying large “X”on touch screenand by generating audio outputat speaker, audio outputincluding a tone having a frequency of 161.00 Hz, creating a C major chord that is audibly out of tune.

32 FIG.J 32 FIG.J 3219 3221 3223 100 3208 111 3218 112 illustrates an embodiment in which the fingering used to play the C major triad is the preferred fingering, based at least on a determination that the fingerprint patterns of contacts,, andincorrespond to the user's left pinky finger (LP), left middle finger (LM), and left thumb (LT), respectively. In response, in accordance with some embodiments, multifunction devicegenerates audio outputat speakercorresponding to a properly tuned C major triad and provides visual feedback indicating that the fingering used to play the chord was the preferred fingering, by displaying staron touch screen.

32 32 FIG.K-L 112 illustrate various embodiments in which the preferred fingering criteria include a criterion that the fingerprint pattern of the activating contact corresponds with the user contacting touch screenwith the tip of their fingertip, as opposed to the pad or entirety of their fingertip.

32 FIG.K 32 FIG.K 100 3225 112 3202 5 3201 3225 100 3208 111 100 3218 112 As illustrated in, multifunction devicedetects contactat a position on touch screencorresponding to the display of activatable “E3” key-of musical keyboard. Because the fingerprint pattern of contactincorresponds to the tip of the user's left middle (LM) fingertip, multifunction devicedetermines that the contact meets the predetermined fingering criteria and generates audio outputat speakerhaving a frequency of 164.81 Hz, corresponding to a properly tuned “E3” tone. Multifunction device, in accordance with some embodiments, also provides visual feedback indicating that the fingering used to play the note was the preferred fingering, by displaying staron touch screen.

32 FIG.L 32 FIG.L 32 FIG.L 100 3227 112 3202 5 3201 3227 100 3206 112 illustrates an embodiment in which the preferred fingering criteria, including a criterion that a tip of the user's finger is used to activate the key, are not met. Multifunction devicedetects contacton touch screenat a position corresponding to display of “E3” key-on musical keyboardin. Upon a determination that contactindoes not meet the preferred fingering criteria, because the fingertip pattern of the contact corresponds to the pad of the user's left middle (LM) fingertip, Multifunction devicedisplays large “X”on touch screen, providing visual feedback indicating that preferred fingering was not user.

32 32 FIGS.M-P illustrate various embodiments in which the preferred fingering criteria further includes a criterion that the user activate a musical key with a finger motion moving backward towards an edge of the musical keyboard.

32 32 FIGS.M-N 32 FIG.M 32 FIG.N 32 FIG.N 32 32 FIGS.M-N 100 3228 112 3202 5 3201 3228 3220 3228 3228 112 3228 112 100 3208 111 100 3218 112 3228 a b As illustrated in, multifunction devicedetects contactat a position on touch screencorresponding to the display of activatable “E3” key-of musical keyboard. Because the fingerprint pattern of contactincludes movementof contactfrom position-on touch screeninto position-on touch screenin, multifunction devicedetermines that the contact meets the predetermined fingering criteria and, in response, generates audio outputat speakerhaving a frequency of 164.81 Hz, corresponding to a properly tuned “E3” tone. Multifunction device, in accordance with some embodiments, also provides visual feedback indicating that the fingering used to play the note was the preferred fingering, by displaying staron touch screenin. The fingerprint pattern of contactinalso corresponds to the tip of the user's left middle (LM) fingertip, further matching preferred fingering criteria in accordance with some embodiments.

32 32 FIGS.O-P 32 32 FIGS.O-P 32 FIG.O 32 FIG.P 32 FIG.P 100 3229 112 3202 5 3201 3229 3230 3229 3229 112 3229 112 100 3206 112 a b As illustrated in, multifunction devicedetects contactat a position on touch screencorresponding to the display of activatable “E3” key-of musical keyboard. Because the fingerprint pattern of contactinincludes movementof contactfrom position-on touch screeninto position-on touch screenin, multifunction devicedetermines that the contact does not meet the predetermined fingering criteria and, in response, provides visual feedback indicating that the fingering used to play the note was not the preferred fingering, by displaying large “X”on touch screenin.

32 32 FIGS.Q-S 100 112 112 3232 27 3232 27 illustrate various embodiments where multifunction devicedetects a fingerprint pattern of a contact at a position on touch screencorresponding to display of an activatable character entry key on a character entry keyboard, and based upon one or more characteristics of the fingerprint pattern, provides feedback to the user if the fingering used to make the contact on touch screendoes not meet a preferred fingering criteria and enters a character (e.g., the letter “c”-) if the fingering used to make the contact does meet a preferred fingering criteria. The preferred fingering including at least a criterion that the user contact “C” key-with their left middle finger.

32 FIG.Q 32 FIG.Q 100 3234 112 3232 27 100 3234 In, multifunction devicedetects contacton touch screenat a position corresponding to display of character entry key-(e.g., the “C” key). Multifunction deviceanalyzes the fingerprint pattern of contactinto determine at least the identity of the user finger corresponding to the contact.

32 FIG.R 32 FIG.R 32 FIG.R 100 3236 100 3206 112 3236 100 3226 53 3224 In some embodiments, as illustrated in, because multifunction devicedetermines that the fingerprint pattern of contactincorresponds to the left index (LI) finger of the user, the contact does not meet the predefined preferred fingering criteria. In response, multifunction devicedisplays large “X”on touch screen, providing visual feedback that the user did not use preferred fingering. In response to detecting contactin, multifunction devicealso displays character “C”-in text box, in accordance with some embodiments.

32 FIG.S 32 FIG.S 100 3238 100 3226 53 3224 In some embodiments, as illustrated in, because multifunction devicedetermines that the fingerprint pattern of contactincorresponds to the left middle (LM) finger of the user, the contact meets the predefined preferred fingering criteria. In response, multifunction devicedisplays character “c”-in text box. In some embodiments, when the contact that activates the character entry key does not meet the preferred fingering criteria, the device does not enter the character corresponding to the character entry key (e.g., the device treats the selection of a character entry key with a non-preferred finger as a failure to activate the key, instead requiring the user to use the preferred finger to activate the character entry key in order to enter the character).

33 33 FIGS.A-C 3 FIG. 1 FIG.A 3300 3300 300 100 3300 are flow diagrams illustrating a methodof training a user on proper fingering techniques in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

3300 As described below, the methodprovides an intuitive way to train a user on proper fingering techniques. The method reduces the cognitive burden on a user when learning proper fingering techniques, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to learn proper fingering techniques faster and more efficiently conserves power and increases the time between battery charges.

100 112 112 3302 3200 3202 5 3201 3232 27 3222 3200 3202 1 3202 24 3201 3240 3232 1 3232 41 3222 32 32 FIGS.A-S 32 32 FIGS.A-S 32 32 FIGS.A-S 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 FIGS.A-P 32 32 FIGS.Q-S In some embodiments, an electronic device (e.g., multifunction devicein) with a touch-sensitive surface (e.g., touch screenin) and a display (e.g., touch screenin) displays () a user interface (e.g., user interfacein) including a respective activatable user interface object (e.g., musical “E3” key-of musical keyboardinand/or character “C” entry key-of QWERTY character entry keyboardin). In some embodiments, the user interface includes a plurality of activatable user interface objects (e.g., a plurality of user interface objects that are capable of being activated). For example, user interfacedisplays a plurality of activatable musical keys-to-of musical keyboardin, and user interfacedisplays a plurality of activatable character entry keys-to-of QWERTY character entry keyboardin.

32 32 FIGS.A-P 32 32 FIGS.A-P 32 32 FIGS.A-P 3304 3202 1 3202 24 3201 In some embodiments, the respective activatable user interface object (e.g., activatable musical “E3” key in) is () one of a plurality of activatable user interface objects that correspond to keys (e.g., activatable musical keys-to-in) in a representation of a musical keyboard (e.g., a virtual piano keyboard such as musical keyboardin).

3232 27 3306 3232 1 3232 41 3222 32 32 FIGS.Q-S 32 32 FIGS.Q-S In some embodiments, the respective activatable user interface object (e.g., activatable character “C” entry key-in) is () one of a plurality of activatable user interface objects that correspond to keys (e.g., activatable character entry keys-to-) in a representation of a character entry keyboard (e.g., a virtual text keyboard, such as QWERTY keyboardin).

100 3308 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 112 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 3202 5 3232 27 112 32 32 FIGS.A-S 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 FIGS.A-S The electronic device (e.g., multifunction device) detects () a fingerprint pattern of a contact (e.g., contact,,,,,,,,,,,,,,, and/orin) on a touch-sensitive surface (e.g., touch screen) that corresponds to a focus selector (e.g., contact,,,,,,,,,,,,,,, and/orin) that is over the respective activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in) on a display (e.g., touch screen). In some embodiments, as illustrated in, the device detects a contact on a touchscreen display over the respective activatable user interface object. In some embodiments, the device detects a contact on a touch sensitive surface, separate from an associated display, at a position corresponding to a position on the associated display displaying a focus selector.

3310 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 3202 5 3232 27 112 100 3312 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 112 3314 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 FIGS.A-S 32 32 FIGS.A-S In response () to detecting the fingerprint pattern of the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) on the touch-sensitive surface that corresponds to the focus selector that is over the respective activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in) on the display (e.g., touch screen): the device (e.g., multifunction device) analyzes () the fingerprint pattern of the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) on the touch-sensitive surface (e.g., touch screen) to determine one or more fingering characteristics of the contact and determines () whether the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) meets predefined preferred fingering criteria based on the one or more fingering characteristics.

3316 3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 3202 5 3232 27 3202 5 3207 3221 3225 3227 3228 3202 5 3205 3217 3232 27 3228 3232 27 3236 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 32 32 FIGS.C,G, andJ-N 32 32 32 32 32 FIGS.B,D-F, andH-I 32 FIG.S 32 FIG.R In some embodiments, the preferred fingering criteria include () a criterion that is met when a finger that corresponds to the fingerprint pattern of the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) used to activate the activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in) matches a predefined preferred finger of a hand for activating the activatable user interface object (e.g., the “j” key on a QWERTY keyboard is preferably activated by the right index finger using standard touch-typing criteria and an “f” key on a QWERTY keyboard is preferably activated by the left index finger using standard touch-typing criteria). For example, a preferred fingering criterion is met inwhen musical “E3” key-is activated by contacts,,,, and/orhaving a fingerprint pattern corresponding to the user's left middle (IM) finger. In contrast, the preferred fingering criterion is not met inwhen musical “E3” key-is activated by contactand/orhaving a fingerprint pattern corresponding to the user's left ring (LR) finger. As another example, a preferred fingering criterion is met inwhen character “C” entry key-is activated by contacthaving a fingerprint pattern corresponding to the user's left middle (LM) finger. In contrast, the preferred fingering criterion is not met inwhen character “C” entry key-is activated by contacthaving a fingerprint pattern corresponding to the user's left index (LI) finger.

3318 3225 3227 3228 3229 112 3202 5 112 3202 5 3225 3228 3229 3202 5 3227 32 32 FIGS.K-P 32 32 FIGS.K-P 32 32 32 FIGS.K andM-P 32 FIG.L In some embodiments, the preferred fingering criteria include () a criterion that is met when the fingerprint pattern of the contact (e.g., contact,,, and/orin) on the touch-sensitive surface (e.g., touch screen) corresponds to an initial angle of elevation between a finger used to activate the activatable user interface object (e.g., musical “E3” key-in) and the display (e.g., touch screen) on which the activatable user interface object is displayed that is within a predefined range of preferred initial angles of elevation for activating the activatable user interface object. (e.g., a key on a musical keyboard is preferably played with the tip of a finger rather than the pad of a finger using standard piano playing guidelines). For example, a preferred fingering criterion is met inwhen musical “E3” key-is activated by contact,, and/orhaving a fingerprint pattern corresponding to the tip of the user's fingertip. In contrast, the preferred fingering criterion is not met iswhen musical “E3” key-is activated by contacthaving a fingerprint pattern corresponding to the pad of the user's fingertip.

3320 3228 3229 112 3202 5 3202 5 3228 112 3201 112 3220 3228 3228 3228 3202 5 3229 112 3201 112 3230 3229 3229 3229 32 32 FIGS.M-P 32 32 FIGS.K-P 32 32 FIGS.M-N 32 FIG.M 32 FIG.N 32 32 FIGS.O-P 32 FIG.O 32 FIG.P a b a b In some embodiments, the preferred fingering criteria include () a criterion that is met when the fingerprint pattern of the contact (e.g., contactand/orin) on the touch-sensitive surface (e.g., touch screen) corresponds to a motion of a finger that activates the activatable user interface object (e.g., musical “E3” key-in) that is within a predefined range of preferred motions for activating the activatable user interface object (e.g., a key on a musical keyboard should be played with the tip of a finger moving backward toward an edge of the keyboard rather than forward into the keyboard, and/or with a predefined intensity). For example, a preferred fingering criterion is met inwhen musical “E3” key-is activated by contacthaving a fingerprint pattern corresponding to motion of the user's fingertip backwards from the initial location of the contact on touch screentowards the front edge of the representation of musical keyboarddisplayed on touch screen(e.g., movementof contactfrom position-into position-in). In contrast, the preferred fingering criterion is not met inwhen musical “E3” key-is activated by contacthaving a fingerprint pattern corresponding to motion of the user's fingertip forwards from the initial location of the contact on touch screenaway from the front edge of the representation of musical keyboarddisplayed on touch screen(e.g., movementof contactfrom position-into position-in).

3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 100 3322 3206 112 3208 111 3212 167 32 32 FIGS.A-S 32 32 32 32 32 32 32 FIGS.B,D,H-I,L,P, andR 32 32 FIGS.E andI 32 FIG.F In accordance with a determination that the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) does not meet the preferred fingering criteria, the device (e.g., multifunction device) provides () feedback indicating that the preferred fingering criteria have not been met (e.g., visual feedback such as large “X”displayed on touch screenin; audio feedback such as audio outputincluding a frequency of 161.00 Hz generated at speakerin; and/or tactile feedback such as tactile outputgenerated, for example, by tactile output generatorin).

100 3324 3202 5 3232 27 3205 3217 3202 5 100 3208 111 3202 5 3206 3202 5 3236 3232 27 100 3226 53 3224 112 3232 27 3206 3232 27 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 FIGS.D andH 32 32 FIGS.D andH 32 FIG.R In some embodiments, in accordance with the determination that the contact does not meet the preferred fingering criteria, the device (e.g., multifunction device) performs () the operation (e.g., plays musical note “E3,” or enters character “C” into an active text box) associated with the activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in). Thus, in some embodiments, even when the preferred fingering criteria are not met, the device still activates the activatable user interface object and performs a corresponding operation, such as playing a note corresponding to a key on the keyboard while providing visual, audio, and/or tactile feedback that the key was “pressed” incorrectly. For example, upon a determination that contactand/ordoes not meet a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction devicegenerates audio outputhaving a frequency of 164.81 (e.g., corresponding to the true frequency of an “E3” tone) at speaker, corresponding to activation of musical “E3” key-in, while providing a different indication that the key was activated using non-preferred fingering (e.g., displaying a large “X”inbecause the user's left index finger was used to activate musical “E3” key-). In another example, upon a determination that contactdoes not meet a predefined fingering criteria including at least a criterion that character “C” entry key-is activated by the user's left middle (LM) finger, multifunction deviceenters (e.g., displays) character “C”-into text boxdisplayed on touch screen, corresponding to activation of character “C” entry key-, while providing a different indication that the key was activated using non-preferred fingering (e.g., displaying a large “X”inbecause the users left index finger was used to activate C” entry key-).

3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 100 3326 3202 5 3232 27 3205 3229 3202 5 100 3208 111 3202 5 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 32 FIGS.B andO-P In some embodiments, in accordance with a determination that the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) does not meet the preferred fingering criteria, the device (e.g., multifunction device) forgoes () performance of the operation (e.g., does not play musical note “E3,” or does not enter character “c” into an active text box) associated with the activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in). Thus, in some embodiments, when the preferred fingering criteria are not met, the device does not activate the activatable user interface object and perform a corresponding operation, such as playing a note corresponding to a key on the keyboard that would be played if the preferred fingering criteria were met. For example, upon a determination that contactand/ordoes not meet a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction deviceforegoes generating audio outputhaving a frequency of 164.81 (e.g., corresponding to the true frequency of an “E3” tone) at speaker, corresponding to activation of musical “E3” key-in.

3328 3205 3217 3202 5 100 3208 111 3202 5 32 32 FIGS.E andI In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes () audible feedback generated by the device (e.g., speakers play warning sound or note sounds “off” as though a musical keyboard had been incorrectly played when fingering criteria for playing a note on the musical keyboard have not been met). For example, upon a determination that contactand/ordoes not meet a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (IM) finger, multifunction devicegenerates audio outputincluding a frequency of 161.00 Hz (e.g., corresponding to an “E3” tone that is out of tune) at speaker, indicating to the user that a preferred fingering was not used to activate musical “E3” key-in. In some embodiments, the audible feedback includes changing a loudness, reverberation or other audible property of the corresponding note instead of or in addition to changing the frequency of the audio output to indicate that the preferred fingering was not used.

3330 3205 3217 3227 3229 3202 5 100 3206 112 3202 5 3236 3232 27 100 3206 112 3232 27 32 32 32 32 32 32 FIGS.B,D,H-I,L, andP 32 FIG.R In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes () visual feedback displayed on the display (e.g., the display of the device flashes or displays highlighting near the contact). For example, upon a determination that contact,,, and/ordoes not meet the a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction devicedisplays large “X”on touch screen, indicating to the user that a preferred fingering was not used to activate musical “E3” key-in. In another example, upon a determination that contactdoes not meet a predefined fingering criteria including at least a criterion that character “C” entry key-is activated by the user's left middle (LM) finger, multifunction devicedisplays large “X”on touch screen, indicating to the user that a preferred fingering was not used to activate character “C” entry key-in.

3332 3205 3202 5 100 3212 167 3202 5 32 FIG.F 32 FIG.F In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes () tactile feedback generated by the device (e.g., the device buzzes or provides some other tactile output indicating that the user's fingering is incorrect). For example, upon a determination that contactindoes not meet a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction devicegenerates tactile output(e.g., using tactile output generator), indicating to the user that a preferred fingering was not used to activate musical “E3” key-in.

3204 3205 3207 3214 3216 3217 3219 3221 3223 3225 3227 3228 3229 3234 3236 3238 100 3334 3202 5 3232 27 3207 3221 3225 3228 3202 5 100 3208 111 3202 5 3238 3232 27 100 3226 53 3224 112 3232 27 32 32 FIGS.A-S 32 32 FIGS.A-P 32 32 FIGS.Q-S 32 32 32 32 32 32 FIGS.C,G,J-K, andM-N 32 FIG.S In accordance with a determination that the contact (e.g., contact,,,,,,,,,,,,,,, and/orin) meets the preferred fingering criteria, the device (e.g., multifunction device) performs () an operation associated with the activatable user interface object (e.g., musical “E3” key-inand/or character “C” entry key-in). For example, the device activates the activatable user interface object without providing feedback indicating that the preferred fingering criteria have not been met. For example, upon a determination that contact,,, and/ormeets a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction devicegenerates audio outputincluding a frequency of 164.81 (e.g., corresponding to the true frequency of an “E3” tone) at speaker, corresponding to activation of musical “E3” key-in. In another example, upon a determination that contactmeets a predefined fingering criteria including at least a criterion that character “C” entry key-is activated by the user's left middle (LM) finger, multifunction deviceenters (e.g., displays) character “c”-into text boxdisplayed on touch screen, corresponding to activation of character “C” entry key-in.

100 3336 3207 3221 3225 3228 3202 5 100 3218 112 3202 5 3228 3232 27 100 3218 112 3232 27 32 32 32 32 FIGS.G,J-K, andN 32 32 32 32 FIGS.G,J-K, andN 32 FIG.S 32 FIG.S In some embodiments, in accordance with a determination that the contact meets the preferred fingering criteria, the device (e.g., multifunction device) provides () feedback indicating that the preferred fingering criteria have been met. (e.g., visual, audio, and/or tactile feedback). For example, upon a determination that contact,,, and/orinmeets a predefined fingering criteria including at least a criterion that musical “E3” key-is activated by the user's left middle (LM) finger, multifunction devicedisplays staron touch screen, indicating to the user that a preferred fingering was used to activate musical “E3” key-in. In another example, upon a determination that contactinmeets a predefined fingering criteria including at least a criterion that character “C” entry key-is activated by the user's left middle (LM) finger, multifunction devicedisplays staron touch screen, indicating to the user that a preferred fingering was used to activate character “C” entry key-in.

33 33 FIGS.A-C 33 33 FIGS.A-C 3300 3300 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, user interface objects, and focus selectors described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, user interface objects, and focus selectors described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

34 FIG. 34 FIG. 3400 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

34 FIG. 3400 3402 3404 3405 3406 3407 3408 3402 3404 3405 3406 3407 3408 3410 3412 3414 3416 3418 3420 As shown in, an electronic deviceincludes a display unitconfigured to display a graphic user interface, a touch-sensitive surface unitconfigured to receive user contacts, optionally a fingerprint sensor unit, an audible feedback unitfor providing audible feedback and a tactile feedback unitfor providing tactile feedback; and a processing unitcoupled to the display unit, the touch-sensitive surface unitand optionally the fingerprint sensor unit, the audible feedback unitand the tactile feedback unit. In some embodiments, the processing unitincludes a display enabling unit, a detecting unit, an analyzing unit, a determining unit, a feedback providing unit, and a performing unit.

3408 3410 3402 3402 3412 3408 3404 3402 3404 3414 3416 3408 3418 3408 3420 The processing unitis configured to enable display of a user interface including a respective activatable user interface object (e.g., with the display enabling unit) and detect a fingerprint pattern of a contact on the touch-sensitive surface unitthat corresponds to a focus selector that is over the respective activatable user interface object on the display unit(e.g., with the detecting unit). The processing unitis also configured to, in response to detecting the fingerprint pattern of the contact on the touch-sensitive surface unitthat corresponds to the focus selector that is over the respective activatable user interface object on the display unit, analyze the fingerprint pattern of the contact on the touch-sensitive surface unitto determine one or more fingering characteristics of the contact (e.g., with the analyzing unit) and determine whether the contact meets predefined preferred fingering criteria based on the one or more fingering characteristics (e.g., with the determining unit). The processing unitis further configured to, in accordance with a determination that the contact does not meet the preferred fingering criteria, provide feedback indicating that the preferred fingering criteria have not been met (e.g., with the feedback providing unit). The processing unitis further configured to, in accordance with a determination that the contact meets the preferred fingering criteria, perform an operation associated with the activatable user interface object (e.g., with the performing unit).

3408 3420 In some embodiments, the processing unitis further configured to, in accordance with the determination that the contact does not meet the preferred fingering criteria, perform the operation associated with the activatable user interface object (e.g., with the performing unit).

3408 3420 In some embodiments, the processing unitis further configured to, in accordance with a determination that the contact does not meet the preferred fingering criteria, forgo performance of the operation associated with the activatable user interface object (e.g., with the performing unit).

3408 3418 In some embodiments, the processing unitis further configured to, in accordance with a determination that the contact meets the preferred fingering criteria, provide feedback indicating that the preferred fingering criteria have been met (e.g., with the feedback providing unit).

In some embodiments, the respective activatable user interface object is one of a plurality of activatable user interface objects that correspond to keys in a representation of a musical keyboard.

In some embodiments, the respective activatable user interface object is one of a plurality of activatable user interface objects that correspond to keys in a representation of a character entry keyboard.

In some embodiments, the preferred fingering criteria include a criterion that is met when a finger that corresponds to the fingerprint pattern of the contact used to activate the activatable user interface object matches a predefined preferred finger of a hand for activating the activatable user interface object.

3402 In some embodiments, the preferred fingering criteria include a criterion that is met when the fingerprint pattern of the contact on the touch-sensitive surface unit corresponds to an initial angle of elevation between a finger used to activate the activatable user interface object and the display uniton which the activatable user interface object is displayed that is within a predefined range of preferred initial angles of elevation for activating the activatable user interface object.

3404 In some embodiments, the preferred fingering criteria include a criterion that is met when the fingerprint pattern of the contact on the touch-sensitive surface unitcorresponds to a motion of a finger that activates the activatable user interface object that is within a predefined range of preferred motions for activating the activatable user interface object.

3400 3406 In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes audible feedback generated by the device(e.g., with the audible feedback unit).

3402 In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes visual feedback displayed on the display unit.

3400 3407 In some embodiments, the feedback indicating that the preferred fingering criteria have not been met includes tactile feedback generated by the device(e.g., with the tactile feedback unit).

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

33 33 FIGS.A-C 1 1 FIGS.A-B 34 FIG. 1 1 FIGS.A-B 3302 3308 3312 3314 3322 3336 3324 3334 3326 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, displaying operation, detecting operation, analyzing operation, determining operation, feedback providing operationsand, performing operationsand, and forgoing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have applications that make use of identity authentication, such as applications that have logins or applications that access private content. A method of identity authentication is fingerprint detection and verification. A device can include a touch-sensitive surface of sufficient sensitivity to detect fingerprints. However, such touch-sensitive surfaces consume more power, and if the device runs on a battery, decreases the time between charges. The embodiments described below include a device that has a touch-sensitive surface that can be sub-divided into regions that can be operated in a mode of enhanced sensitivity that is sufficient to detect fingerprints or in a mode of reduced sensitivity. A region of the touch-sensitive surface is operated in the enhanced-sensitivity mode if one or more criteria are met, and is otherwise operated in the reduced-sensitivity mode. This enhances the sensitivity of portions of the touch-sensitive surface on an as-needed basis. By operating portions of the touch-sensitive surface in the enhanced-sensitivity mode on an as-needed basis, power consumption is reduced while the device remains capable of fingerprint detection, thereby providing a convenient and efficient user interface that conserves battery power.

450 451 100 112 167 35 35 36 36 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 112 450 451 169 451 450 1 FIG.A 4 FIG.B 4 FIG.B 35 35 36 36 FIGS.A-J andA-B 35 35 FIGS.A-J 35 35 FIGS.A-J In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-J andA-B optionally includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, cither an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and optionally one or more fingerprint sensorsin response to detecting the inputs described inon the touch-sensitive surfacewhile displaying the user interfaces shown inon the display.

35 FIG.A 4 FIG.A 35 FIG.A 3501 112 100 3501 112 3502 3502 1 424 3502 2 426 3502 3 428 3502 4 444 3502 5 446 3502 6 416 3502 7 418 112 illustrates user interfacedisplayed on touch screenof a device (e.g., device). User interfaceincludes one or more application icons, as described above with reference to. Touch screenincludes one or more regions, each of which corresponds to a respective application icon. As shown in, region-corresponds to “Messages” icon. Region-corresponds to “Calendar” icon. Region-corresponds to “Photos” icon. Region-corresponds to “Notes” icon. Region-corresponds to “Settings” icon. Region-corresponds to “Phone” icon. Region-corresponds to “Mail” icon. In some embodiments, the regions correspond to quadrants (or other geometric divisions) of the touch sensitive surface (e.g., touch screen).

100 3502 100 3502 100 3502 112 100 3502 100 3502 112 3502 100 100 3502 100 100 3502 112 112 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A Deviceoperates a respective region (e.g., one of regionsin) in a reduced-sensitivity mode or an enhanced-sensitively mode. When deviceoperates a respective region (e.g., one of regionsin) operating in enhanced-sensitivity mode, deviceis capable of detecting the presence/absence of a contact in the respective region (e.g., one of regionsin) on touch screenand movement of the contact, and also is capable of detecting fingerprint features of the contact, such as minutia features that enable the contact to be uniquely identified as a previously registered fingerprint. When deviceoperates the respective region (e.g., one of regionsin) in reduced-sensitivity mode, deviceis capable of detecting the presence/absence of the contact in the respective region (e.g., one of regionsin) on touch screenand movement of the contact, but is not capable of detecting fingerprint features of the contact. Thus, for example, when a contact is located in a respective region (e.g., one of regionsin) of devicethat is being operated in enhanced-sensitivity mode, deviceis capable of detecting fingerprint features, which can be used for identity authentication or authorized access, and/or detection of fine movement of the contact. On the other hand, when a contact is located in a respective region (e.g., one of regionsin) operating in reduced-sensitivity mode, deviceis not capable of detecting fingerprint features of the contact. In some embodiments, deviceindependently operates and transitions each respective region (e.g., one of regionsin) between reduced-sensitivity mode and enhanced-sensitivity mode. In some embodiments, the a majority of touch screenor, optionally the entire touch screen, is capable of being operated in the enhanced-sensitivity mode of operation.

3502 100 3502 3502 3502 100 3502 100 3502 3502 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A When there is no contact detected on a respective region (e.g., one of regionsin), deviceoperates the respective region (e.g., one of regionsin) in reduced-sensitivity mode. Operating a respective region (e.g., one of regionsin) in reduced-sensitivity mode consumes less power than operating the same respective region (e.g., one of regionsin) in enhanced-sensitivity mode. When devicedetects a contact in a respective region (e.g., one of regionsin) that is operating in reduced-sensitivity mode, deviceoperates the respective region (e.g., one of regionsin) in enhanced-sensitivity mode if the contact meets one or more fingerprint-sensor activation criteria, and otherwise continues to operate the respective region (e.g., one of regionsin) in reduced-sensitivity mode.

3502 3502 35 FIG.A 35 FIG.A In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a fingerprint-shaped contact is detected in the respective region (e.g., one of regionsin). Thus, for example, a fingerprint-shaped contact detected in the respective region (e.g., one of regionsin) would activate the enhanced-sensitivity mode, but a stylus contact would not. In some embodiments, a single contact gesture activates the enhanced-sensitivity mode of operation, while a multi-contact gesture does not activate the enhanced-sensitivity mode of operation.

3502 3502 112 112 112 100 35 FIG.A 35 FIG.A In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a fingerprint-shaped contact is detected to be moving or rotating (e.g., twisting) at a speed below a predetermined threshold speed (or at a rotation speed below a predetermined threshold). For example, a fingerprint that is slowly moving through or twisting in a respective region (e.g., one of regionsin) activates the enhanced-sensitivity mode for the respective region (e.g., one of regionsin). In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a contact is detected on touch screenwhile a focus selector corresponding to the contact is over a fingerprint-sensitive user interface element. In some embodiments, on touch screen, the focus selector corresponding to the contact is the contact itself. Thus, for example, this criterion is met when a contact is detected on touch screenover a fingerprint-sensitive user interface element. An example of a fingerprint-sensitive user interface element is an application icon whose corresponding application is configured to require authentication or authorization for access. In some embodiments, whether an application requires authentication or authorization for access is configurable by a user of device. Another example of a fingerprint-sensitive user interface element is an interactive user interface object (e.g., a virtual dial or knob).

3502 3502 3502 3502 112 100 3502 3502 35 FIG.A 35 FIG.A 35 FIG.A 35 FIG.A In some embodiments, when the device operates a respective region (e.g., one of regionsin) in enhanced-sensitivity mode, the device operates the other respective regionsin reduced-sensitivity mode. Operating only one respective region (e.g., one of regionsin) in enhanced-sensitivity mode consumes less power than operating multiple respective regionsin enhanced-sensitivity mode. In some embodiments, when the contact no longer meets the criteria, or the contact is removed from the respective region (e.g., by liftoff of the contact from touch screen), devicereverts to operating the respective region (e.g., one of regionsin) in reduced-sensitivity mode. In some embodiments, the reversion to operating the respective region (e.g., one of regionsin) in reduced-sensitivity mode occurs after a predetermined amount of time after the contact is removed or no longer meets the criteria (e.g., a timeout).

35 FIG.A 35 FIG.A 35 FIG.B 35 FIG.C 3504 418 418 3502 7 100 140 418 3504 418 100 3502 7 3502 7 100 3504 100 3504 140 100 3506 140 100 3504 140 100 140 3508 140 3508 3512 3510 shows a contact with fingerprintdetected on “Mail” icon. “Mail” iconcorresponds to region-, which is being operated by devicein reduced-sensitivity mode. In, e-mail client module, to which “Mail” iconcorresponds, is configured to require authentication or authorization for access. In response to detecting the contact with fingerprinton “Mail” icon, devicestarts to operate region-in enhanced-sensitivity mode. With region-operating in enhanced-sensitivity mode, deviceis capable of detecting fingerprint features of fingerprintfor purposes of authorization. In some embodiments, if devicedetermines that fingerprintdoes not correspond to a previously registered fingerprint or does not belong to an authorized user of e-mail client module, devicedisplays messagealerting the user of the lack of authorization to access e-mail client module, as shown in. If devicedetermines that fingerprintcorresponds to a previously registered fingerprint that belongs to an authorized user of e-mail client module, deviceactivates email client moduleand displays user interfaceof e-mail client module, as shown in. User interfaceincludes, for example, a listing of email accountsand a listing of corresponding inboxes.

100 3508 140 3512 3510 3502 7 In some embodiments, a user of devicecan have access to some email accounts (and corresponding inboxes) and not others. When user interfaceof e-mail client moduleis displayed, email accounts listingand inboxes listingonly includes the email accounts and inboxes the user is authorized to access, based on the fingerprint detected in region-.

35 FIG.D 100 100 112 112 3516 3518 3516 3518 100 3516 3518 112 3520 100 3520 3520 illustrates devicein a locked state. While deviceis in the locked state and touch screenis active (e.g., not in sleep mode), a lock screen is displayed on touch screen. The lock screen includes unlock objectand virtual groove (or unlock image), with unlock objectlocated at one end (e.g., the left end) of virtual groove (or unlock image). Devicecan be unlocked by dragging unlock objectto the opposite end (e.g., the right end) of virtual groove. Touch screenalso includes regionthat deviceoperates in reduced-sensitivity mode or enhanced-sensitivity mode. When there is no contact detected in region, regionis operated in reduced-sensitivity mode.

3514 3516 3514 3518 112 3516 3518 3516 3518 3514 3520 3514 3514 3514 3520 35 35 FIGS.E-F 35 FIG.G a b A contact with fingerprinton unlock object, and movement of the contact and fingerprinttoward the right end of virtual groove, are detected on touch screen. In response to detecting the movement of the contact, unlock objectmoves toward the right end of virtual groove, as shown inand in; the contact drags unlock objecttoward the right end of virtual groove. Before fingerprintreaches region(e.g., while fingerprintis at location-or-), regionremains in reduced-sensitivity mode.

3514 3520 3514 3514 100 3520 100 3520 3514 3520 100 3514 100 3514 3514 3522 100 100 3514 3524 100 3514 3520 100 100 3514 3524 c 35 FIG.F 35 FIG.G 35 FIG.G 35 FIG.D When the device detects fingerprintin region(e.g., when the contact with fingerprinthas moved to location-), as shown inand in, deviceoperates regionin enhanced-sensitivity mode. In some embodiments, deviceoperates regionin enhanced-sensitivity mode if the contact with fingerprintis detected moving in regionbelow a predetermined speed threshold. Devicedetects one or more fingerprint features of fingerprint. Based on the detected fingerprint features, devicedetermines whether fingerprintmeets one or more unlock criteria or not. In the example shown in, if fingerprintmeetsunlock criteria (e.g., the fingerprint is a previously registered fingerprint of a user of device), then deviceis unlocked and, for example, an application launch user interface (or other unlocked interface) is displayed. In contrast, if fingerprintdoes not meetunlock criteria (e.g., the fingerprint is not identified as a previously registered fingerprint of a user of device) or if the features of fingerprintcannot be detected (e.g., because regionis in reduced-sensitivity mode), then deviceremains in the locked state and optionally displays an alternative user interface for unlocking device(e.g., a passcode entry user interface). Alternatively, when fingerprintdoes not meetunlock criteria, the lock screen shown inis redisplayed.

35 FIG.H 3526 112 3526 3528 3530 3526 3530 3528 112 3530 3530 3528 3530 shows user interfacedisplayed on touch screen. User interfaceincludes numeral value fieldand virtual knob. User interfaceis a user interface of an application that includes numerical value input and/or manipulation (e.g., a calculator, a thermostat control application, a unit converter, a spreadsheet). A user interacts with virtual knobto adjust a value displayed in numerical value field. For example, a user places a fingerprint on touch screenover virtual knoband twists the fingerprint to turn virtual knob. The value in numerical value fieldchanges with the turning of virtual knob.

35 35 FIGS.H-J 35 FIG.I 3530 112 100 3530 100 112 3530 3530 100 112 3530 3532 3530 3532 3532 100 112 3530 112 3530 100 3532 3532 100 3532 3530 3532 3528 3530 In, the area of virtual knobis a region of touch screenthat deviceoperates in reduced-sensitivity mode or enhanced-sensitivity mode. When there is no fingerprint detected on virtual knob, deviceoperates a portion of touch screenthat includes virtual knobin reduced-sensitivity mode. When a fingerprint is detected on virtual knob, deviceoperates a portion of touch screenthat includes virtual knobin either reduced-sensitivity mode or enhanced-sensitivity mode based on the rate at which the detected fingerprint twists. For example,illustrates fingerprintdetected on virtual knob. The user twists fingerprintat a rate above a predetermined threshold speed. In accordance with the above-threshold twisting rate of fingerprint, deviceoperates a portion of touch screenthat includes virtual knobin reduced-sensitivity mode. While operating the portion of touch screenthat includes virtual knobin reduced-sensitivity mode, devicedetects motion (e.g., twisting) of fingerprintbut not the features of fingerprint. Thus, devicedetects twisting of fingerprintin relatively coarse increments. Virtual knobrotates in coarse increments in accordance with the detected coarse twisting increments of fingerprint. The value in numerical value fieldchanges in coarse increments in accordance with the coarse increments of rotation of virtual knob.

35 FIG.J 3534 3532 3530 3534 3534 100 112 3530 112 3530 100 3532 3532 100 3532 3530 3530 3528 3530 100 112 3530 3534 3530 illustrates fingerprint, analogous to fingerprint, detected on virtual knob. The user twists fingerprintat a rate below the predetermined threshold speed. In accordance with the below-threshold twisting rate of fingerprint, deviceoperates the portion of touch screenthat includes virtual knobin enhanced-sensitivity mode. While operating the portion of touch screenthat includes virtual knobin enhanced-sensitivity mode, devicedetects features of fingerprintas well as its twisting motion. By detecting the features of fingerprint, deviceis able to detect the movement of those features and thus detect fine movement, including twisting in fine increments, of fingerprint. Virtual knobrotates in fine increments in accordance with the detected fine twisting increments of fingerprint. The value in numerical value fieldchanges in fine increments accordance with the fine increments of rotation of virtual knob. In some embodiments, deviceoperates the portion of touch screenthat includes virtual knobin enhanced-sensitivity mode in response to detecting fingerprinttwisting in virtual knob, without regard to the speed of the twisting.

35 35 FIGS.A-J 35 FIG.A 35 FIG.A 112 100 112 451 100 300 112 451 112 3502 3520 3530 3501 112 3502 444 438 3502 5 444 112 444 438 As shown in, touch screenincludes one or more regions that deviceoperates in reduced-sensitivity mode or enhanced-sensitivity mode. It should be appreciated that, in some embodiments, any portion of touch screen(or touch-sensitive surface) is capable of being operated by the device (e.g., deviceor) in reduced-sensitivity mode or enhanced-sensitivity mode. In some embodiments, for a given displayed user interface, some portions of touch screen(or touch-sensitive surface), such as the portions of touch screenoutside of regions, region, or virtual knob, remain in reduced-sensitivity mode; the capability to transition those portions between reduced-sensitivity mode and enhanced-sensitivity mode is disabled. For example, for user interfaceas shown in, the capability to transition the portions of touch screenoutside of regionsbetween reduced-sensitivity mode and enhanced-sensitivity mode is disabled. Also, in some embodiments, a region where the transitioning between sensitivity modes is enabled moves along with the corresponding user interface object or affordance. For example, in, if the locations of “Notes” iconand “Weather” iconare swapped with each other, region-moves to the new location of “Notes” icon, and the portion of touch screencorresponding to the old location of “Notes” icon(i.e., the new location of “Weather” icon) is disabled from transitioning between reduced-sensitivity mode and enhanced-sensitivity mode.

112 112 112 112 112 In some embodiments, the portions of the touch-sensitive surface that are transitioned between the reduced-sensitivity mode and enhanced-sensitivity mode correspond to different groups of sensors that can be enabled and disabled separately. For example, a first set of sensors corresponding to a lower right quadrant of touch screen, a second set of sensors corresponding to an upper right quadrant of touch screen, a third set of sensors corresponding to a lower left quadrant of touch screen, and a fourth set of sensors corresponding to an upper left quadrant of touch screencan each be independently transitioned between the reduced-sensitivity mode and the enhanced-sensitivity mode. In some embodiments, the different regions are selected to be transitioned between the reduced-sensitivity mode and the enhanced-sensitivity mode based on a location of a contact or a user interface object on touch screen, as described in greater detail above. In situations where the device determines that a region of the user interface that needs enhanced-sensitivity data spans multiple regions of sets of sensors, the device, optionally, enables all of the multiple regions of sets of sensors to operate in the enhanced-sensitivity mode of operation, while one or more other sets of sensors continue to operate in the reduced-sensitivity mode of operation.

36 36 FIGS.A-B 3 FIG. 1 FIG.A 3600 3600 300 100 3600 are flow diagrams illustrating a methodof operating a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

3600 As described below, the methodprovides an intuitive way to operate a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation. The method reduces the cognitive burden on a user when operating a touch-sensitive surface, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling operation of a portion of a touch-sensitive surface in an enhanced-sensitivity mode of operation conserves power and increases the time between battery charges.

3602 3504 3502 7 3514 3514 3520 3532 3530 3603 3502 1 3502 7 3502 1 3502 7 112 35 FIG.A 35 FIG.F 35 FIG.H 35 FIG.A c The device detects () a contact in a first region of the touch-sensitive surface, where the first region of the touch-sensitive surface is in a reduced-sensitivity mode of operation., for example, illustrates a contact with fingerprintdetected in region-that is being operated in reduced-sensitivity mode.shows a contact with fingerprintmoving to location-, which is in regionthat is being operated in reduced-sensitivity mode.shows a contact with fingerprintdetected in virtual knob, which is also a region that is being operated in reduced-sensitivity mode. In some embodiments, the touch-sensitive surface includes () a plurality of different regions (e.g., regions that correspond to subsets of the touch-sensitive surface), and the device is configured to independently transition each of the plurality of different regions between the reduced-sensitivity mode and the enhanced-sensitivity mode (e.g., the device can turn the enhanced-sensitivity mode “on” for one or more of the regions without turning the enhanced-sensitivity mode on for the whole touch-sensitive surface). Turning the enhanced-sensitivity mode on in certain areas, but not in others, if there is no need to have the whole touch-sensitive surface in enhanced-sensitivity mode, reduces demands on the processor and extends battery life. For example,shows regions-through-. In some embodiments, the device is configured to independently transition each of regions-through-(or regions of touch screenthat include these regions) between reduced-sensitivity mode and enhanced-sensitivity mode.

3604 3606 100 3504 3514 3532 3534 In response to detecting the contact (), the device determines () whether the contact meets one or more fingerprint-sensor activation criteria. Devicedetermines, for example, whether fingerprint(or fingerprintoror) meets one or more fingerprint-sensor activation criteria.

3608 3504 3502 7 3514 3520 In some embodiments, the one or more fingerprint-sensor activation criteria include () a criterion that is met when a fingerprint-shaped contact is detected in the first region of the touch-sensitive surface. For example, a fingerprint-sensor activation criterion is met by detection of a contact with fingerprintin region-(or detection of a contact with fingerprintin region).

3610 3534 3530 3514 3520 In some embodiments, the one or more fingerprint-sensor activation criteria include () a criterion that is met when a fingerprint-shaped contact is detected moving at a speed below a predetermined threshold speed (e.g., enhanced-sensitivity helps make slow movement more accurate/slow movement indicates that the user is interacting with a fingerprint sensitive user interface element). For example, a fingerprint-sensor activation criterion is met by detection of fingerprinttwisting in virtual knobat a rate below a predetermined threshold. As another example, a fingerprint-sensor activation criterion is met by detection of a contact with fingerprintmoving in regionat a rate below a predetermined threshold.

3612 3534 3530 3504 3502 7 In some embodiments, the one or more fingerprint-sensor activation criteria include () a criterion that is met when a contact is detected on the touch-sensitive surface while a focus selector corresponding to the contact is over a fingerprint-sensitive user interface element. For example, the device detects a contact over a control that operates in accordance with a detected fingerprint (e.g., a knob or dial that operates in response to twisting of a fingerprint, or an affordance that is restricted so that it is only activated in accordance with a detected fingerprint) and turns on the fingerprint sensitivity in a region of the touch-sensitive surface that includes the contact and/or the control so that the fingerprint corresponding to the contact can be identified and used to operate the control. For example, a fingerprint-sensor activation criterion is met by detection of fingerprinttwisting in virtual knob. As another example, a fingerprint-sensor activation criterion is met by detection of a contact with fingerprintin region-.

3614 100 3502 7 3520 112 3530 In accordance with a determination that the contact meets the fingerprint-sensor activation criteria, the device operates () the first region of the touch-sensitive surface in an enhanced-sensitivity mode of operation (e.g., for contacts within the first region of the touch-sensitive surface, the device can detect the presence/absence and movement of the contact, and, in addition, the device can detect fingerprint features of the contact such as minutia features that enable the contact to be uniquely identified as a previously registered fingerprint). For example, deviceoperates region-(and similarly regionor a portion of touch screenthat includes virtual knob) in enhanced-sensitivity mode when the fingerprint-sensor activation criteria are met.

3618 3532 100 3530 35 FIG.I In accordance with a determination that the contact does not meet the fingerprint-sensor activation criteria, the device continues to operate () the first region of the touch-sensitive surface in the reduced-sensitivity mode of operation (e.g., for regions of the touch-sensitive surface that are operating in the reduced-sensitivity mode, the device can detect the presence/absence and movement of the contact, but is not able to detect fingerprint features of the contact such as minutia features that would enable the contact to be uniquely identified as a previously registered fingerprint). For example, in, when the fingerprint-sensor activation criteria includes movement or rotation below a threshold rate, when fingerprinttwists above the threshold speed, the devicecontinues to operate virtual knobin reduced-sensitivity mode.

3616 3620 100 3502 7 3520 3530 100 3504 3514 3534 100 3502 7 3520 3530 100 In some embodiments, while the first region of the touch-sensitive surface is in the enhanced-sensitivity mode of operation, the device is capable () of detecting fingerprint details of a contact detected in the first region; and while the first region of the touch-sensitive surface is in the reduced-sensitivity mode of operation, the device is not capable () of detecting fingerprint details of a contact detected in the first region. For example, while deviceis operating region-(or regionor) in enhanced-sensitivity mode, deviceis capable of detecting fingerprint details (e.g., fingerprint features) of a contact (e.g., a contact with fingerprint, a contact with fingerprint, a contact with fingerprint) detected in the region. While deviceis operating region-(or regionor) in reduced-sensitivity mode, deviceis not capable of detecting fingerprint details (e.g., fingerprint features) of a contact detected in the region and thus does not track movements of the contact as precisely as when the region is operating in the enhanced-sensitivity mode.

3622 3624 3626 100 3504 3502 7 3506 3508 140 3504 100 3514 3520 100 100 3514 3522 3524 100 3534 3530 3530 35 35 FIGS.B-C 35 FIG.G 35 FIG.J In some embodiments, while operating the first region of the touch-sensitive surface in the enhanced-sensitivity mode (), the device detects () fingerprint features of the contact, and, in response to detecting the fingerprint features of the contact, performs () an operation in accordance with the fingerprint features (e.g., in response to detecting the fingerprint features of the contact, the device performs an identity-based operation or an operation that requires detecting fingerprint features of the contact). For example, the device detects small movements of the contact based on detecting movements of fingerprint minutia features, where the small movements of the contact would not be detectable based on movement of the overall contact without looking at the fingerprint features. In, devicedetects the features of fingerprintin region-and either displays error messageor displays user interfaceof e-mail client modulebased on the detected features of fingerprint. In, devicedetects the features of fingerprintin regionand unlocks deviceor leaves devicebased on whether the fingerprintmeets unlock criteriaor does not meet the unlock criteria. In, devicedetects movement of features of fingerprintin virtual knoband rotates virtual knobin fine increments in accordance with the detected movement of the fingerprint features.

3622 3628 3502 112 3502 7 3502 112 35 FIG.A In some embodiments, while operating the first region in the enhanced-sensitivity mode (), the device operates () one or more of the other regions of the touch-sensitive surface in the reduced-sensitivity mode (e.g., while the first region is enabled to detect fingerprint details of contacts detected on the touch-sensitive surface, other regions of the touch-sensitive surface are able to detect the presence/absence and movement of contacts, without detecting fingerprint details of the contacts). For example,shows multiple regionsthat are operable in reduced-sensitivity mode or enhanced-sensitivity mode (and the remainder of touch screenis operable in reduced-sensitivity mode). While region-is operated in enhanced-sensitivity mode, the other regionsare operated in reduced-sensitivity mode. Maintaining some of the regions of the touch screenin reduced-sensitivity mode while operating a respective region in enhanced-sensitivity mode provides the benefits of enhanced precision and/or ability to authenticate a user in the respective region while reducing power usage by keeping other regions in a lower power reduced-sensitivity mode when the enhanced-sensitivity mode is not needed for the other regions.

3630 3502 7 3502 7 In some embodiments, operating () the first region of the touch-sensitive surface in the enhanced-sensitivity mode consumes more power than operating the first region of the touch-sensitive surface in the reduced-sensitivity mode. Thus, switching the first region of the touch-sensitive surface from reduced-sensitivity mode to enhanced-sensitivity mode on an “as needed” basis reduces the overall energy consumption of the device, thereby increasing energy efficiency and battery life of the device. For example, operating region-in enhanced-sensitivity mode when a contact is detected within, and in reduced-sensitivity mode at other times reduces power consumption compared to operating region-in enhanced-sensitivity mode all the time.

3632 3502 3502 7 3502 35 FIG.A In some embodiments, operating () more than the first region of the touch-sensitive surface in the enhanced-sensitivity mode consumes more power than operating only the first region of the touch-sensitive surface in the enhanced-sensitivity mode (e.g., while operating the rest of the touch-sensitive surface in the reduced-sensitivity mode). Thus, operating the rest of the touch-sensitive surface in the reduced-sensitivity mode while operating the first region of the touch-sensitive surface in the enhanced-sensitivity mode reduces the overall energy consumption of the device, thereby increasing energy efficiency and battery life of the device. For example, operating just whichever regionin which a contact is detected (e.g., region-in) in enhanced-sensitivity mode and the rest in reduced-sensitivity mode reduces power consumption compared to operating more than one of regionsin enhanced-sensitivity mode.

36 36 FIGS.A-B 36 36 FIGS.A-B 3600 3600 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, and user interfaces described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, and user interfaces described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

37 FIG. 37 FIG. 3700 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

37 FIG. 3700 3702 3704 3708 3702 3704 3708 3710 3712 3714 3716 3718 As shown in, an electronic deviceincludes a display unit, a touch-sensitive surface unitconfigured to receive contacts, and a processing unitcoupled to the display unitand the touch-sensitive surface unit. In some embodiments, the processing unitincludes a detecting unit, a determining unit, an operating unit, a performing unit, and a transitioning unit.

3708 3704 3710 3704 3712 3704 3714 3704 3714 The processing unitis configured to: detect a contact in a first region of the touch-sensitive surface unit(e.g., with the detecting unit), where the first region of the touch-sensitive surface unitis in a reduced-sensitivity mode of operation; in response to detecting the contact: determine whether the contact meets fingerprint-sensor activation criteria (e.g., with the determining unit); in accordance with a determination that the contact meets the fingerprint-sensor activation criteria, operate the first region of the touch-sensitive surface unitin an enhanced-sensitivity mode of operation (e.g., with the operating unit); and in accordance with a determination that the contact does not meet the fingerprint-sensor activation criteria, continue to operate the first region of the touch-sensitive surface unitin the reduced-sensitivity mode of operation (e.g., with the operating unit).

3704 3708 3710 3704 3708 3710 In some embodiments, while the first region of the touch-sensitive surface unitis in the enhanced-sensitivity mode of operation, the processing unitis capable of detecting fingerprint details of a contact detected in the first region (e.g., with the detecting unit), and while the first region of the touch-sensitive surface unitis in the reduced-sensitivity mode of operation, the processing unitis not capable of detecting fingerprint details of a contact detected in the first region (e.g., with the detecting unit).

3708 3704 3710 3716 In some embodiments, the processing unitis configured to, while operating the first region of the touch-sensitive surface unitin the enhanced-sensitivity mode: detect fingerprint features of the contact (e.g., with the detecting unit), and in response to detecting the fingerprint features of the contact, perform an operation in accordance with the fingerprint features (e.g., with the performing unit).

3704 3708 3718 In some embodiments, the touch-sensitive surface unitincludes a plurality of different regions, and the processing unitis configured to independently transition each of the plurality of different regions between the reduced-sensitivity mode and the enhanced-sensitivity mode (e.g., with the transitioning unit).

3708 3704 3714 In some embodiments, the processing unitis configured to, while operating the first region in the enhanced-sensitivity mode, operate one or more of the other regions of the touch-sensitive surface unitin the reduced-sensitivity mode (e.g., with the operating unit).

3704 3704 In some embodiments, operating the first region of the touch-sensitive surface unitin the enhanced-sensitivity mode consumes more power than operating the first region of the touch-sensitive surface unitin the reduced-sensitivity mode.

3704 3704 In some embodiments, operating more than the first region of the touch-sensitive surface unitin the enhanced-sensitivity mode consumes more power than operating only the first region of the touch-sensitive surface unitin the enhanced-sensitivity mode.

3704 In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a fingerprint-shaped contact is detected in the first region of the touch-sensitive surface unit.

In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a fingerprint-shaped contact is detected moving at a speed below a predetermined threshold speed.

3704 In some embodiments, the one or more fingerprint-sensor activation criteria include a criterion that is met when a contact is detected on the touch-sensitive surface unitwhile a focus selector corresponding to the contact is over a fingerprint-sensitive user interface element.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

36 36 FIGS.A-B 1 1 FIGS.A-B 37 FIG. 1 1 FIGS.A-B 3602 3606 3614 3618 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operation, determining operation, and operating operationsandare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have respective applications for performing respective operations, along with respective corresponding user interfaces and affordances. Typically, in order to have an operation performed by a device, the user launches the corresponding application (e.g., by activating a corresponding application icon) on the device so that the corresponding user interfaces and affordances are displayed. The user then activates the operation using the user interface and affordances. However, sometimes the user wants an operation performed in the moment while interacting with an unrelated application or user interface. In current methods, the user would still have to activate the application icon to launch the application, in order to perform the operation. This is time consuming and detracts from the user experience. The embodiments described below improve on these methods by associating a particular fingerprint with an operation on a device. When the user performs a gesture with that fingerprint while a user interface unrelated to the desired operation is displayed, the device performs the operation. Thus, the user can activate an operation while a user interface unrelated to the operation is displayed; the user has the ability to activate the operation quickly and efficiently, thereby improving the speed and efficiency of the user interface.

450 451 100 112 167 38 38 39 39 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 112 450 451 169 451 450 1 FIG.A 4 FIG.B 4 FIG.B 38 38 39 39 FIGS.A-P andA-E 38 38 FIGS.A-P 38 38 FIGS.A-P In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-P andA-E optionally includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, cither an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screenand optionally one or more fingerprint sensors; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and optionally one or more fingerprint sensorsin response to detecting the inputs described inon the touch-sensitive surfacewhile displaying the user interfaces shown inon the display.

38 FIG.A 38 FIG.A 3801 112 100 112 3801 112 illustrates user interfacedisplayed on touch screenof a device (device). Touch screenincludes an integrated fingerprint sensor. In, application launch user interfacethat includes one or more icons for launching respective applications is displayed on touchscreen display. At the device, one or more respective fingerprints are associated with respective operations. In some embodiments, the fingerprints are differentiated by hand and finger. For example, a right thumbprint is associated with a respective function or operation, and a left thumbprint is associated with another respective function or operation. In some embodiments, a camera-related function or operation (e.g., a shutter function for capturing a photo or video, displaying a camera preview) is associated with a respective fingerprint.

3801 112 3802 112 3802 3802 3804 143 112 3806 3802 112 3804 3806 3806 164 38 FIG.B While application launch user interfaceis displayed on touch screen, the device detects a gesture with fingerprinton touch screen. Fingerprintis identified by the device as a right thumbprint. In this example, the right thumbprint is associated with display of a camera interface and a camera preview, and still image or photo capture. In response to detecting the gesture with fingerprint, the device displays camera interface, corresponding to camera module, on touch screen, as shown in. In some embodiments, the device displays camera previewin response to detecting fingerprinton touch screenfor at least a predefined amount of time. Camera interfaceincludes camera preview. Camera previewshows content that is detected by a camera (e.g., optical sensor(s)) on the device, and previews what will be captured by the camera as a still image or video if a media capture function is activated on the device.

112 3802 3802 3802 3802 In some embodiments, fingerprints detected on touch screenare identified based on comparison to previously registered features of fingerprints of users or to features associated with predefined fingerprint types. For example, fingerprintis identified as a right thumbprint through identification of fingerprintas the right thumbprint of a particular user or identification of fingerprintas a generic (i.e., without identifying that fingerprintbelongs to a particular user) right thumbprint.

3802 3802 112 3802 3806 3808 143 3804 3806 3801 38 FIG.C 38 FIG.C 38 FIG.D The device detects a continuation of the gesture with fingerprint, where the continuation includes an increase in the intensity of a contact corresponding to fingerprintabove a predefined intensity threshold (e.g., an intensity threshold that is higher than a standard contact-detection intensity threshold), as shown in. For example, the user presses down on touch screenwith the contact corresponding to fingerprint. In response to detecting the intensity increase above the threshold, the device activates a shutter function, which activates a capturing of a still image or photo corresponding to camera preview. In some embodiments, the device displays an animation of representationof the captured photo moving to a film strip or camera roll for camera application, as illustrated in. After the photo is captured, the device ceases displaying camera interfaceand camera preview, and re-displays the previously displayed user interface (e.g., application launch user interface), as shown in.

3802 3802 38 FIG.B In some embodiments, the device activates the shutter function for capturing content in response to detecting fingerprintcontinuously for longer than a respective time threshold (e.g., 2, 5, 10 seconds or any other reasonable time threshold), rather than in response to detecting an increase in the contact intensity. For example, the shutter function is activated if fingerprintinis continuously detected, from initial detection, for longer than the time threshold. Thus, in some embodiments, even while a user interface that doesn't include an affordance for displaying a camera interface or activating a shutter function (or another operation) is displayed, a user can still activate display of a camera interface and/or a shutter function (or the another operation) with a gesture that includes a fingerprint associated with the respective operation(s).

38 38 FIGS.E-H 38 FIG.E 38 FIG.A 38 FIG.F 38 FIG.G 38 FIG.H 3801 112 3810 112 3810 3810 3804 3812 112 3812 3813 112 3810 112 3810 112 3804 3812 3801 illustrates another example of activating a respective operation while a user interface that does not include an affordance for activating the respective operation is displayed.illustrates application launch user interfacedisplayed on touch screen, as in. A gesture with fingerprintis detected on touch screen. Fingerprintis detected by the device to be a right thumbprint. In this example, the right thumbprint is associated with display of a camera interface and a camera preview, and video recording. In response to detecting the gesture with fingerprint, the device displays camera interfaceand camera previewon touch screen, as shown in. Also, the device activates recording of video corresponding to camera preview. While video is being recorded, recording indicatoris, optionally, displayed on touch screento indicate that recording is in progress as well as the time length of the in-progress recording.shows the gesture with fingerprintno longer detected on touch screendue to, for example, fingerprinthaving been lifted off touch screen. In response to detecting the liftoff, the device ceases recording the video and ceases displaying camera interfaceand camera preview, and re-displays user interface, as shown in.

3802 112 112 112 3814 3802 3810 112 3816 3802 3810 38 FIG.I 38 FIG.J In some embodiments, the gesture that includes a fingerprint is specific with respect to orientation, location, and/or duration, and whether an operation associated with the fingerprint is activated depends on whether the fingerprint meets the orientation, location, and or duration requirements. For example, in some embodiments, if fingerprintis detected on touch screenfor less than a predefined time period, the operation is not performed. In some embodiments, if the fingerprint is detected at a location other than a predetermined location (e.g., the upper right quadrant of touch screenbased on the current screen orientation) on touch screen(e.g., fingerprintbeing in the lower right quadrant (), as opposed to fingerprintorbeing in the upper right quadrant), the operation is not performed. In some embodiments, if the fingerprint is detected at an orientation other than a predetermined orientation (e.g., 45°±10° degree angle from right-side-up vertical) on touch screen(e.g., fingerprintbeing outside of the orientation angle range (), as opposed to fingerprintorbeing within the orientation angle range), the operation is not performed (e.g., media is not capture, and the device does not replace display of a currently displayed user interface with the media capture user interface).

38 38 FIGS.K-L 38 FIG.K 38 FIG.L 3818 112 3820 112 112 3820 3820 3822 112 illustrates yet another example of activating a respective operation while a user interface that does not include an affordance for activating the respective operation is displayed.illustrates user interfacefor a notes application displayed on touch screen. A gesture with fingerprintis detected on touch screen, and then lifted off touch screenafter a predefined time period. Fingerprintis detected by the device to be a right thumbprint. In this example, the right thumbprint is associated with display of a communications interface, such as a phone application user interface, email application user interface, or a messaging application user interface. In response to detecting the gesture with fingerprintand the subsequent liftoff, the device displays phone application interfaceon touch screen, as shown in.

38 38 FIGS.M-P 38 FIG.M 38 FIG.N 3824 112 3801 112 3824 3824 3804 3826 3826 3804 3826 3801 In some embodiments, multiple fingerprints are respectively associated on the device with different operations. For example, the right thumbprint is associated with still image capture and the left thumbprint is associated with video recording, an example of which is shown in.illustrates a gesture with fingerprintdetected on touch screenwhile application launch user interfaceis displayed on touch screen. Fingerprintis detected to be a right thumbprint. In response to detecting the gesture with fingerprint, camera interfaceand camera previeware displayed and a photo corresponding to camera previewis captured, as shown in. After the photo is captured, the device, optionally, ceases to display camera interfaceand camera preview, and re-displays the previously displayed user interface (e.g., application launch user interface).

38 FIG.O 38 FIG.P 3828 112 3801 112 3828 3828 3804 3830 3830 3804 3830 3828 3804 3830 3801 illustrates a gesture with fingerprintdetected on touch screenwhile user interfaceis displayed on touch screen. Fingerprintis detected to be a left thumbprint. In response to detecting the gesture with fingerprint, camera interfaceand camera previeware displayed and video corresponding to camera previewis captured, as shown in. Video recording and display of camera interfaceand camera previewis, optionally, stopped when liftoff of the gesture with fingerprintis detected. After camera interfaceand camera previewceases to be displayed, the device re-displays user interface.

39 39 FIGS.A-E 3 FIG. 1 FIG.A 3900 3900 300 100 3900 are flow diagrams illustrating a methodof performing operations associated with fingerprint gestures in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

3900 As described below, the methodprovides an intuitive way to perform operations associated with fingerprint gestures. The method reduces the cognitive burden on a user when performing operations associated with fingerprint gestures, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to perform operations associated with fingerprint gestures faster and more efficiently conserves power and increases the time between battery charges.

3902 38 38 FIGS.A-D The device associates () a first fingerprint with a first operation (e.g., during a fingerprint registration process assign the right thumbprint to a shutter function of a camera application). For example, in, the right thumbprint is associated with a camera shutter function.

3904 3801 112 3818 3906 3801 3818 38 38 FIGS.A andE 38 FIG.K 38 38 FIG.A orE 38 FIG.K The device displays (), on the display, a respective user interface that includes affordances for performing a plurality of operations other than the first operation (e.g., the user interface is not a user interface for performing the first operation and the user interface does not contain affordances or other user interface elements for performing the first operation). User interfacedisplayed on touch screen, as shown in, for example, includes affordances for launching applications (e.g., application icons) but not an affordance for capturing a photo or recording video. Similarly, user interface() includes affordances related to note-taking (e.g., respective affordances for opening an existing note, creating a new note, deleting a note, etc.) but not an affordance related to making a phone call. In some embodiments, the respective user interface does not include () a displayed control for performing the first operation. For example, user interface() does not include a control for capturing a photo or recording video. User interface() does not include a control for making a phone call or for activating display of a phone application interface.

3908 3818 153 143 3910 3801 143 38 FIG.K 38 38 FIG.A orE In some embodiments, the respective user interface is () a user interface of a first application that is not configured to perform the first operation, and the first operation is performed by a second application different from the first application. For example, user interface() is a user interface for a notes application (e.g., notes module), which is not configured to perform camera operations. Camera operations are performed by a camera application (e.g., camera module). In some embodiments, the respective user interface is () a user interface of an application launch screen that is not configured to perform the first operation, and the first operation is performed by a respective application different from the application launch screen. For example, user interface() is an application launch screen not configured to perform camera operations. Camera operations are performed by a camera application (e.g., camera module).

3912 100 3802 112 100 3810 112 100 3820 112 38 FIG.A 38 FIG.E 38 FIG.K While displaying the respective user interface that includes affordances for performing the plurality of operations other than the first operation, the device detects () a first gesture that includes detecting the first fingerprint on the touch-sensitive surface (e.g., a tap gesture, a tap and drag gesture, a touch and hold gesture, or other predefined gesture associated with performing the first operation)., for example, shows devicedetecting a gesture that includes fingerprinton touch screen.shows devicedetecting a gesture that includes fingerprinton touch screen.shows devicedetecting a gesture that includes fingerprinton touch screen.

3914 3802 3802 3802 3804 3806 3816 3816 38 38 FIGS.A-C 38 FIG.J In some embodiments, the first gesture is an orientation-specific gesture, and detecting the orientation-specific gesture includes () detecting the first fingerprint on the touch-sensitive surface at a predetermined orientation. For example, the operation is only performed when the device detects a gesture performed with the thumb while the thumb is at a predefined angle (e.g., at 45° compared to a primary axis of the touch-sensitive surface) or within a predefined range of angles (e.g., at 45°±5°, 10°, or 15° compared to a primary axis of the touch-sensitive surface) relative to a primary axis of the touch-sensitive surface. In contrast, in some embodiments, if the first fingerprint is not at the predefined angle or within the predefined range of angles, then the first operation is not performed (e.g., no operation is performed, or an operation associated with a currently displayed icon (or other user interface element) at the location of the first fingerprint is performed instead of the first operation). For example, in, the gesture with fingerprintis, optionally, orientation-specific, and fingerprintis detected to be within a predefined range of angles (e.g., 45°±10°) with respect to an axis parallel to a long edge of the device. In response to detecting the gesture with fingerprint, camera interfaceand camera previeware displayed. On the other hand, fingerprint() is outside of the predefined angle range, and no camera interface or camera preview is displayed in response to detection of fingerprint.

3916 3802 3802 112 3802 3804 3806 3810 3810 38 38 FIGS.A-C 38 FIG.I In some embodiments, the first gesture is a location-specific gesture, and detecting the location-specific gesture includes () detecting the first fingerprint on the touch-sensitive surface at a predetermined location. For example, the operation is only performed when the device detects a gesture performed with the thumb while the thumb is at a predetermined location (e.g., the upper right hand corner of the touch-sensitive surface) on the touch-sensitive surface. In contrast, in some embodiments, if the first fingerprint is not at the predetermined location, then the first operation is not performed (e.g., no operation is performed, or an operation associated with a currently displayed icon (or other user interface element) at the location of the first fingerprint is performed instead of the first operation). For example, in, the gesture with fingerprintis, optionally, location-specific, and fingerprintis detected to be within the predetermined location (e.g., upper right quadrant of touch screenbased on current screen orientation). In response to detecting the gesture with fingerprint, camera interfaceand camera previeware displayed. On the other hand, fingerprint() is outside of the predetermined location, and no camera interface or camera preview is displayed in response to detection of fingerprint.

3918 3802 3802 112 3802 3804 3806 3802 112 38 38 FIGS.A-C In some embodiments, the first gesture is a duration-specific gesture, and detecting the duration-specific gesture includes () detecting the first fingerprint on the touch-sensitive surface for at least a predetermined amount of time. For example, the operation is only performed when the device detects a gesture performed with a right thumb that is present on the touch-sensitive surface for more than a predefined time period (e.g., 0.05, 0.1, 0.2, 0.5 seconds, or some other reasonable time period). In contrast, in some embodiments, if the first fingerprint is not detected for at least the predetermined amount of time, then the first operation is not performed (e.g., no operation is performed, or an operation associated with a currently displayed icon (or other UI element) at the location of the first fingerprint is performed instead of the first operation). For example, in, the gesture with fingerprintis, optionally, duration-specific, and fingerprintis detected on touch screenfor more than the predefined time period (e.g., 0.2 seconds). In response to detecting the gesture with fingerprint, camera interfaceand camera previeware displayed. On the other hand, in some embodiments, if fingerprintis detected on touch screenfor less than the predefined time period, no camera interface or camera preview is displayed in response.

3920 112 112 112 112 38 38 FIGS.A-P In some embodiments, the touch-sensitive surface is () a touchscreen display with an integrated fingerprint sensor. The devices shown inhave a touch screen (e.g., touch screen) as the touch-sensitive surface on which gestures are detected. The touch screen has, optionally, an integrated fingerprint sensor. The integrated fingerprint sensor optionally spans the area of touch screen. In some embodiments touch screenhas a sufficiently high resolution touch sensing capability to detect ridges of fingerprints and the touch screencan be used as a fingerprint sensor.

3922 3802 3802 3802 In some embodiments, the first fingerprint is a previously registered fingerprint (e.g., the first fingerprint is registered as a right thumb of user X, with the registration being stored in the memory of the device), and detecting the first gesture includes () identifying a detected fingerprint as the first fingerprint based on a comparison between detected features of the detected fingerprint and previously registered features of the first fingerprint (e.g., the fingerprint is identified as being a thumb by matching the first fingerprint to a previously registered fingerprint of a particular user that was identified by the user as being a thumb fingerprint). For example, identification of fingerprintinvolves comparing fingerprintto previously registered fingerprints of users and identifying fingerprintas the particular fingerprint (e.g., right thumbprint) of a particular user.

3924 3802 3802 3802 In some embodiments, the first fingerprint is a predefined type of fingerprint (e.g., a “thumbprint”), and detecting the first gesture includes () identifying a detected fingerprint as the first fingerprint based on a comparison between features of the detected fingerprint and features associated with the predefined type of fingerprint (e.g., the first fingerprint is identified as being a generic “thumb” without specifically identifying that the fingerprint is a particular fingerprint of a particular user). For example, identification of fingerprintinvolves comparing fingerprintto fingerprint features that are not user-specific (e.g., size) and identifying fingerprintas a generic finger (e.g., generic right thumb, generic left index finger, etc.).

3926 3928 3804 3806 3802 3826 3824 38 FIG.B 38 FIG.N In response to detecting the first gesture (), the device performs () the first operation. For example, in, camera interfaceand camera previewis displayed in response to the device detecting the gesture with fingerprint. As another example, in, a photo corresponding to camera previewis captured in response to the device detecting the gesture with fingerprint.

3930 3824 3804 3806 3802 38 FIG.N 38 FIG.A In some embodiments, the first operation is performed () in response to detecting the first fingerprint on the touch-sensitive surface (e.g., the first operation is performed in response to detecting the first fingerprint on the touch-sensitive surface). For example, the photo capture operation inis performed by the device in response to the device detecting fingerprint. As another example, display of camera interfaceand camera previewinis performed by the device in response to the device detecting fingerprint. In some embodiments, the first operation is performed in response to detecting the first fingerprint on the touch-sensitive surface for more than a predetermined amount of time.

3932 3822 3820 38 FIG.L In some embodiments, the first operation is performed () in response to detecting liftoff of the first fingerprint from the touch-sensitive surface (e.g., the first operation is performed in response to detecting liftoff of the first fingerprint from the touch-sensitive surface after detecting the first fingerprint on the touch-sensitive surface). For example, the display of phone application interfaceinis performed by the device in response to the device detecting liftoff of fingerprint. In some embodiments, the first operation is performed in response to detecting liftoff of the first fingerprint from the touch-sensitive surface after detecting the first fingerprint on the touch-sensitive surface for more than a predetermined amount of time.

3934 3824 3936 3810 3938 3822 3820 3940 3804 3802 38 38 FIGS.M-N 38 38 FIGS.E-F 38 38 FIGS.K-L 38 38 FIGS.A-B In some embodiments, the first operation includes () taking a photo. For example, the device incaptures a photo in response to detecting fingerprint. In some embodiments, the first operation includes () recording video. For example, the device inrecords video in response to detecting fingerprint. In some embodiments, the first operation includes () displaying a communication interface (e.g., a user interface for a phone, email or other electronic messaging application that is a default application associated with the first fingerprint, or is an application associated with the first fingerprint by a user of the device). For example, the device indisplays phone application interfacein response to detecting fingerprint. In some embodiments, the first operation includes () displaying a camera interface (e.g., a user interface for a camera application for capturing photos or video). For example, the device indisplays camera interfacein response to detecting fingerprint.

3942 3804 3804 164 38 38 FIGS.B-C In some embodiments, while displaying () the camera interface, the device determines whether media-capture criteria have been met. In accordance with a determination that the media-capture criteria have been met, the device starts to capture media with a camera. In accordance with a determination that the media-capture criteria have not been met, the device forgoes capturing media with a camera. For example, in, the device displays camera interface. While displaying camera interface, the device determines whether one or more media capture criteria are met. If the criteria are met, the device starts capturing media (e.g., one or more photos, video) with a camera (e.g., optical sensor(s)). If the criteria are not met, the device doesn't capture media.

3944 3802 112 38 FIG.C In some embodiments, the media-capture criteria include () a criterion that is met when a contact that corresponds to the first fingerprint has an intensity above a respective intensity threshold (e.g., the user presses down harder with the fingerprint to take a photo or start taking video while the camera preview is displayed). For example, fingerprintinhas an intensity above a respective intensity threshold (e.g., an intensity threshold that is greater than a contact-detection intensity threshold that corresponds to detecting a contact on the touch screen display), and thus meets the criteria. In response, the device captures a photo.

3946 3802 38 FIG.C In some embodiments, the media-capture criteria include () a criterion that is met when a contact that corresponds to the first fingerprint has been continuously detected for longer than a respective time threshold (e.g., the user maintains the contact on the touch-sensitive surface for a time that is longer than the respective time threshold to take a photo or to start taking video while the camera preview is displayed). For example, in, the media-capture criterion is, optionally, a fingerprint detection duration criterion rather than an intensity criterion. The duration criterion is met if fingerprintis continuously detected for longer than a respective time threshold.

3948 3806 3802 38 FIG.B In some embodiments, in response to detecting the first fingerprint for at least a predefined amount of time, the device displays () a camera preview that includes a preview of media that will be captured by a camera. For example, inthe device displays camera previewin response to detecting fingerprintfor at least a predefined amount of time (e.g., 0.05, 0.1, 0.2, 0.5 seconds, or some other reasonable time period).

3950 3952 3954 3956 3958 In some embodiments, while displaying the camera preview, the device detects () an input corresponding to a request to capture media corresponding to the camera preview (e.g., the device detects liftoff of the first fingerprint, an increase in intensity of the fingerprint over the respective intensity threshold, or activation of a physical button associated with capturing a picture). In response to detecting () the input corresponding to the request to capture media corresponding to the camera preview, the device captures () a still image corresponding to the camera preview, ceases () to display the camera preview, and redisplays () the respective user interface. For example, in response to detecting the fingerprint on the touch-sensitive surface, the device displays a camera preview. In response to detecting liftoff of the fingerprint while displaying the camera preview, the device takes a photo and ceases to display the camera preview. Thus, while a user is in the middle of using the device for a respective task (e.g., reading news, checking email, or some other task), the user is able to quickly transition to photo capture mode, take a photo, and then return to the respective task with minimal interruption.

38 38 FIGS.B-D 3802 3806 3806 3806 3806 3804 3801 For example,illustrate the intensity of fingerprintincreasing above a respective intensity threshold while camera previewis displayed. The intensity increase is a request to the device to capture media corresponding to camera preview. In response to detecting the request, the device captures a photo corresponding to camera preview, ceases displaying camera preview(along with ceasing to display camera interface), and redisplays user interface.

3960 3812 3812 3810 38 FIG.F In some embodiments, while displaying the camera preview, the device captures () media (e.g., video or a series of still photos taken at predetermined intervals such as in a burst photo-capture mode) corresponding to the camera preview. In some embodiments, the device starts to capture the media in response to detecting the first fingerprint on the touch-sensitive surface. In some embodiments, the device starts to capture the media in response to detecting a subsequent capture-initiation input such as detecting the first fingerprint on the touch sensitive surface for more than a predetermined amount of time or detecting an increase in intensity of the first fingerprint above a respective intensity threshold. For example,shows the device recording a video corresponding to camera preview, while displaying camera preview, in response to detecting fingerprint.

3962 3964 3966 3968 3970 3812 3810 3812 3804 3801 38 FIG.F 38 FIG.G 38 FIG.H In some embodiments, while capturing the media corresponding to the camera preview (e.g., after taking one or more photos in a series of photos or while capturing video), the device detects () an input corresponding to a request to cease capturing the media (e.g., the device detects liftoff of the first fingerprint, detects an increase in intensity of the fingerprint over the respective intensity threshold, or detects activation of a physical button associated with ceasing to capture the media, or detects a tap or press and hold gesture on the touch-sensitive surface or a portion of the touch-sensitive surface corresponding to the a media capture icon). In response to detecting () the input corresponding to the request to cease capturing the media, the device ceases () to capture the media, ceases () to display the camera preview, and redisplays () the respective user interface. In some embodiments, in response to detecting the fingerprint on the touch-sensitive surface, the device displays a camera preview. In response to continuing to detect the fingerprint on the touch-sensitive surface for more than a predetermined amount of time, the device starts taking video, and in response to detecting liftoff of the fingerprint while displaying the camera preview, the device stops taking video and ceases to display the camera preview. Alternatively, in response to detecting the fingerprint on the touch-sensitive surface, the device displays a camera preview and in response to continuing to detect the fingerprint on the touch-sensitive surface for more than a predetermined amount of time, the device takes a series of multiple photos in a burst photo-capture mode, and in response to detecting liftoff of the fingerprint while displaying the camera preview, the device ceases to display the camera preview. Thus, while a user is in the middle of using the device for a respective task (e.g., reading news, checking email, or some other task), the user is able to quickly transition to video capture mode, take a video or a burst of photos, and then return to the respective task with minimal interruption. While video corresponding camera previewis being recorded (), for example, the device detects liftoff of fingerprint. In response, the device stops recording the video (), ceases displaying camera preview(along with ceasing to display camera interface), and redisplays user interface().

3972 3976 3978 3980 3801 3824 3828 38 38 FIGS.M-P 38 FIG.M In some embodiments, the device associates () a second fingerprint different from the first fingerprint with a second operation different from the first operation (e.g., during a fingerprint registration process assign the right thumbprint to a shutter function of a camera application). The device displays (), on the display, the respective user interface, where the respective user interface does not include an affordance for performing the second operation (e.g., the user interface is an application launch screen or a user interface for a different application that is not a user interface for performing the second operation and the user interface does not contain affordances or other user interface elements for performing the second operation). While displaying the respective user interface, the device detects () a second gesture that includes detecting the second fingerprint on the touch-sensitive surface. In response to detecting the gesture that includes the second fingerprint on the touch-sensitive surface, the device performs () the second operation. For example, in, the right thumbprint is associated with photo capture, and the left thumbprint is associated with video recording. The device displays user interface(), which does not include affordances for photo capture or video recording. The device captures a photo in response to detecting fingerprint, which is a right thumbprint, and records video in response to detecting fingerprint.

3974 38 38 FIGS.M-P In some embodiments, the first operation is () an operation associated with capturing still photos (e.g., displaying a still camera application user interface or capturing a photo), and the second operation is an operation associated with capturing video (e.g., displaying a video camera application user interface or taking a video). For example, when the user places a left thumbprint anywhere on the touch-sensitive surface (e.g., touchscreen) of the device, the device launches a still camera application, and when the user places a right thumbprint anywhere on the touch-sensitive surface (e.g., touchscreen) of the device, the device launches a video camera application. In some embodiments, the first operation and the second operation launch the same application but in different modes (e.g., a camera application is launched either in a still image capture mode if the device detects the gesture performed with the left thumbprint or a video capture mode if the device detects the gesture performed with the right thumbprint. For example, in, the right thumbprint is associated with photo capture, and the left thumbprint is associated with video recording.

39 39 FIGS.A-E 39 39 FIGS.A-E 3900 3900 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, gestures described, and user interfaces above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, gestures, and user interfaces described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

40 FIG. 40 FIG. 4000 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

40 FIG. 4000 4002 4004 4006 4008 4002 4004 4008 4010 4012 4014 4016 4018 4022 4024 4026 As shown in, an electronic deviceincludes a display unitconfigured to display a respective user interface that includes affordances for performing a plurality of operations other than a first operation, a touch-sensitive surface unitconfigured to receive gestures, optionally a fingerprint sensor unitfor detecting fingerprints, and a processing unitcoupled to the display unitand the touch-sensitive surface unit. In some embodiments, the processing unitincludes an associating unit, a detecting unit, a performing unit, a determining unit, a starting unit, a display enabling unit, a capturing unit, and a ceasing unit.

4008 4010 4022 4004 4012 4014 The processing unitis configured to: associate a first fingerprint with the first operation (e.g., with the associating unit); enable display of a respective user interface that includes affordances for performing a plurality of operations other than the first operation (e.g., with the display enabling unit); while enabling display of the respective user interface that includes affordances for performing the plurality of operations other than the first operation, detect a first gesture that includes detecting the first fingerprint on the touch-sensitive surface unit(e.g., with the detecting unit); and in response to detecting the first gesture, perform the first operation (e.g., with the performing unit).

4004 In some embodiments, the first operation is performed in response to detecting the first fingerprint on the touch-sensitive surface unit.

4004 In some embodiments, the first operation is performed in response to detecting liftoff of the first fingerprint from the touch-sensitive surface unit.

4004 In some embodiments, the first gesture is an orientation-specific gesture, and detecting the orientation-specific gesture includes detecting the first fingerprint on the touch-sensitive surface unitat a predetermined orientation.

4004 In some embodiments, the first gesture is a location-specific gesture, and detecting the location-specific gesture includes detecting the first fingerprint on the touch-sensitive surface unitat a predetermined location.

4004 In some embodiments, the first gesture is a duration-specific gesture, and detecting the duration-specific gesture includes detecting the first fingerprint on the touch-sensitive surface unitfor at least a predetermined amount of time.

In some embodiments, the respective user interface does not include a displayed control for performing the first operation.

In some embodiments, the respective user interface is a user interface of a first application that is not configured to perform the first operation, and the first operation is performed by a second application different from the first application.

In some embodiments, the respective user interface is a user interface of an application launch screen that is not configured to perform the first operation, and the first operation is performed by an respective application different from the application launch screen.

In some embodiments, the first operation includes taking a photo.

In some embodiments, the first operation includes recording video.

In some embodiments, the first operation includes enabling display of a communication interface.

In some embodiments, the first operation includes enabling display of a camera interface.

4008 4016 4018 4024 4024 In some embodiments, the processing unitis configured to, while enabling display of the camera interface: determine whether media-capture criteria have been met (e.g., with the determining unit); in accordance with a determination that the media-capture criteria have been met, start to capture media with a camera (e.g., with the starting unitor the capturing unit); and in accordance with a determination that the media-capture criteria have not been met, forgo capturing media with a camera (e.g., with the capturing unit).

In some embodiments, the media-capture criteria include a criterion that is met when a contact that corresponds to the first fingerprint has an intensity above a respective intensity threshold.

In some embodiments, the media-capture criteria include a criterion that is met when a contact that corresponds to the first fingerprint has been continuously detected for longer than a respective time threshold.

4008 4022 In some embodiments, the processing unitis configured to, in response to detecting the first fingerprint for at least a predefined amount of time, enable display of a camera preview that includes a preview of media that will be captured by a camera (e.g., with the display enabling unit).

4008 4012 4024 4026 4022 In some embodiments, the processing unitis configured to: while enabling display of the camera preview, detect an input corresponding to a request to capture media corresponding to the camera preview (e.g., with the detecting unit); and in response to detecting the input corresponding to the request to capture media corresponding to the camera preview: capture a still image corresponding to the camera preview (e.g., with the capturing unit), cease enabling display of the camera preview (e.g., with the ceasing unit), and re-enable display of the respective user interface (e.g., with the display enabling unit).

4008 4024 4012 4026 4026 4022 In some embodiments, the processing unitis configured to: while enabling display of the camera preview, capture media corresponding to the camera preview (e.g., with the capturing unit); while capturing the media corresponding to the camera preview, detect an input corresponding to a request to cease capturing the media (e.g., with the detecting unit); and in response to detecting the input corresponding to the request to cease capturing the media: cease to capture the media (e.g., with the ceasing unit), cease enabling display of the camera preview (e.g., with the ceasing unit), and re-enable display of the respective user interface (e.g., with the display enabling unit).

4004 In some embodiments, the touch-sensitive surface unitis a touchscreen display with an integrated fingerprint sensor.

In some embodiments, the first fingerprint is a previously registered fingerprint, and detecting the first gesture includes identifying a detected fingerprint as the first fingerprint based on a comparison between detected features of the detected fingerprint and previously registered features of the first fingerprint.

In some embodiments, the first fingerprint is predefined type of fingerprint, and detecting the first gesture includes identifying a detected fingerprint as the first fingerprint based on a comparison between features of the detected fingerprint and features associated with the predefined type of fingerprint.

4008 4010 4002 4022 4004 4012 4004 4014 In some embodiments, the processing unitis configured to: associate a second fingerprint different from the first fingerprint with a second operation different from the first operation (e.g., with the associating unit); enable display, on the display unit, of the respective user interface (e.g., with the display enabling unit), wherein the respective user interface does not include an affordance for performing the second operation; while enabling display of the respective user interface, detect a second gesture that includes detecting the second fingerprint on the touch-sensitive surface unit(e.g., with the detecting unit); and in response to detecting the gesture that includes the second fingerprint on the touch-sensitive surface unit, perform the second operation (e.g., with the performing unit).

In some embodiments, the first operation is an operation associated with capturing still photos, and the second operation is an operation associated with capturing video.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

39 39 FIGS.A-E 1 1 FIGS.A-B 40 FIG. 1 1 FIGS.A-B 3912 3928 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detection operationand performing operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have graphical user interfaces with a respective control for the user interface. The respective control (e.g., a color palate for an image editing application) is displayed on the user interface in response to a user toolbar or menu window selection with, for example, a cursor controlled by a mouse (or other peripheral device). The device described below improves on existing methods by displaying on a display a respective control for a user interface in response to detecting on a touch-sensitive surface a first fingerprint associated with moving the respective control. The first fingerprint corresponds to a focus selector at a first location on the display that does not include the respective control.

450 451 100 112 167 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 359 1 359 2 112 450 451 359 1 359 2 359 1 359 2 450 1 FIG.A 41 41 42 42 FIGS.A-K andA-C 4 FIG.B 4 FIG.B 41 41 42 42 FIGS.A-K andA-C 41 41 FIGS.A-K 41 41 FIGS.A-K In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toincludes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, cither an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screenand fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-); in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and an integrated fingerprint sensor-or a separate fingerprint sensor-in response to detecting the inputs described inon integrated fingerprint sensor-or separate fingerprint sensor-, while displaying the user interfaces shown inon display.

41 41 FIG.A-K 41 41 41 41 FIGS.B-F andH-K 100 4101 112 112 100 112 100 4104 4101 illustrate portable multifunction devicedisplaying a video game on user interfaceof touch screen. In some embodiments, touch screencomprises a fingerprint sensor such that deviceis enabled to detect a fingerprint at any position on touch screen.further illustrate devicedisplaying the video game in a pause mode indicated by pause symboldisplayed on user interface.

41 FIG.A 41 41 FIGS.E-H 41 FIG.D 41 FIG.A 4102 4101 4102 4102 4102 4102 4102 4102 4112 4106 4102 4106 100 4102 a a b a illustrates displaying respective control(e.g., a directional pad, d-pad or joypad for directional control of an interactive object of the video game) for user interfaceat prior location-. Prior location-is different from first location-in. Respective controlis displayed at prior location-prior to detecting a first fingerprint (e.g., a user's right pinky finger) associated with moving respective control(e.g., fingerprintdetected in). In some embodiments, the first fingerprint corresponds to a finger that is not typically associated with manipulation of user interface objects (e.g., a user's ring or pinky finger).further illustrates detecting fingerprint(e.g., a user's right index finger) over respective control. In response to detecting fingerprint, deviceperforms operations associated with respective control.

41 FIG.B 4108 4102 112 4102 4102 4102 a illustrates detecting first fingerprint(e.g., a user's right pinky finger) associated with moving respective controlat a first location on touch screenwhile displaying respective controlat prior location-. The first location is different from the prior location and does not include respective control.

41 FIG.C 4110 112 4102 112 4110 100 4102 4110 4102 4101 illustrates detecting fingerprinton touch screenwhile respective controlis not displayed on touch screen. In response to detecting fingerprint, deviceforgoes displaying respective controlin accordance with a determination that fingerprintis a fingerprint (e.g., a user's right index finger) different from the first fingerprint (e.g., a user's right pinky finger) and is not associated with moving respective controlfor user interface.

41 FIG.D 4112 4102 4101 4112 112 4112 4112 112 4112 112 4102 a a illustrates detecting first fingerprint(e.g., a user's right pinky finger) associated with moving respective controlfor user interfaceat first position-on touch-sensitive surface. First fingerprintcorresponds to a focus selector (e.g., a centroid of a respective contact associated with fingerprintdetected on the touch screen) at first location-on touch screenthat does not include respective control.

41 FIG.F 41 FIG.D 4102 4102 112 4112 4112 a a illustrates displaying respective controlat first location-on touch screenin response to detecting first fingerprint(e.g., a user's right pinky finger) at first position-in.

41 FIG.F 4112 112 4112 4112 4102 a b illustrates detecting a dragging gesture of first fingerprint(e.g., a user's right pinky finger) on touch screenfrom first location-to second location-. The second location is different from the first location and does not include respective control.

41 FIG.G 41 FIG.E 41 FIG.G 41 FIG.E 41 FIG.G 4112 4112 4102 4102 112 4112 4112 4114 4102 4114 100 4102 a b a illustrates detecting liftoff of first fingerprintfrom location-in.also illustrates maintaining display of respective controlat first location-on touch screenafter detecting liftoff of first fingerprintfrom location-in.further illustrates detecting fingerprint(e.g., a user's right index finger) over respective control. In response to detecting fingerprint, deviceperforms operations associated with respective control.

41 FIG.H 4116 4102 4102 112 4102 4102 b b illustrates detecting respective fingerprint(e.g., a user's right middle finger) at a second location while displaying respective controlat first location-on touch screen. The second location is different from first location-and does not include respective control.

41 FIG.I 4118 4102 4102 112 4102 4102 b b illustrates detecting respective fingerprint(e.g., a user's right pinky finger) at a second location while displaying respective controlat first location-on touch screen. The second location is different from first location-and does not include respective control.

41 FIG.J 41 FIG.F 41 FIG.I 41 FIG.F 41 FIG.I 4102 4102 4102 4102 112 4102 4102 112 4112 4112 4118 4112 4118 4118 4112 4102 4102 112 4118 4102 4102 112 b c c b b c c illustrates ceasing to display respective controlat first location-and displaying respective controlat second location-on touch screen. Respective controlis displayed at second location-on touch screenin response to detecting respective fingerprintat second location-in, or fingerprintin, wherein the respective fingerprintoris the first fingerprint. For example, when the respective fingerprintis dragged to the second location-in, respective controlis displayed at second location-on touch screen. In another example, when the device detects touchdown of the respective fingerprintat the second location in, respective controlis displayed at the second location-on touch screen.

41 FIG.K 41 FIG.H 4102 4102 112 4116 4116 4102 4101 b illustrates maintaining display of respective controlat first location-on touch screenin response to detecting respective fingerprintat the second location inand in accordance with a determination that respective fingerprintis a second fingerprint (e.g., a user's right middle finger) that is different from the first fingerprint (e.g., a user's right pinky finger) and is not associated with moving respective controlfor user interface.

42 42 FIGS.A-C 3 FIG. 1 FIG.A 4200 4200 300 100 4200 are flow diagrams illustrating a methodof displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display, a touch-sensitive surface and a fingerprint sensor. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

4200 As described below, the methodprovides an intuitive way to display a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control. The method reduces the cognitive burden on a user when displaying a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to display a respective control for a user interface based on detection of a first fingerprint associated with moving the respective control faster and more efficiently conserves power and increases the time between battery charges.

4202 100 4101 112 41 41 FIG.A-K An electronic device with a display, a touch-sensitive surface, and a fingerprint sensor displays () a user interface on the display., for example, show portable multifunction devicedisplaying user interfaceincluding a video game on touch screen.

4204 100 4102 4101 4102 4101 4102 4102 4102 4102 100 4108 100 4102 4102 4102 4102 4108 41 FIG.A 41 41 FIGS.A-B 41 41 FIGS.E-F 41 FIG.B 41 FIG.E 41 FIG.B a a b a b In some embodiments, (immediately) prior to detecting a first fingerprint associated with moving a respective control for the user interface, the device displays () the respective control at a prior location that is different from a first location, and in response to detecting the first fingerprint, the device ceases to display the respective control at the prior location (e.g., the device moves the respective control from the prior location to a new location in response to detecting the first fingerprint)., for example, shows devicedisplaying respective controlfor user interfaceat prior location-, prior to detecting a first fingerprint (e.g., a user's right pinky) associated with moving respective interface. Prior location-of respective controlinis different from first location-of respective controlin., for example, shows devicedetecting first fingerprintat a first location., for example, shows deviceceasing to display respective controlat prior location-and displaying respective controlat first location-in response to detecting first fingerprintat the first location in.

4206 100 4102 4106 4102 4102 4102 100 4103 4101 4106 4102 41 FIG.A a In some embodiments, when the respective control is displayed at the prior location, the device responds () to inputs corresponding to the prior location by performing operations associated with the respective control. Additionally, in some embodiments, when the respective control is displayed at the prior location, the device responds to inputs corresponding to the first location by performing operations (e.g., scrolling, selecting, etc.) that are not associated with the respective control (or, optionally by not performing any operations if the inputs do not correspond to other operations in the user interface)., for example, shows deviceperforming operations associated with respective controlin response to detecting fingerprint(e.g., a user's right index finger) over respective controlwhile respective controlis displayed at prior location-. In one example, devicechanges direction of an interactive object of the video game (e.g., vehicle) on user interfacein accordance with the location of fingerprintover respective controlcorresponding to a downward direction.

4208 100 4112 4112 112 4102 100 4102 112 41 FIG.D 41 FIG.D a The device detects () the first (predetermined) fingerprint associated with moving (e.g., relocating on the display) the respective control for the user interface on the touch-sensitive surface, where the first fingerprint corresponds to a focus selector at a first location on the display that does not include the respective control., for example, shows devicedetecting first fingerprint(e.g., a user's right pinky finger) at first location-on touch screenthat does not include respective control. In, for example, deviceis not displaying respective controlon touch screen. In some embodiments, the respective control corresponds to a hidden control for a user interface (e.g., directional controls for a video game, audio controls for a media playback application, a color palate or cropping tools for an image editing application, navigation controls for a web browser application, etc.).

4210 100 4102 112 41 41 41 41 FIGS.A-B andE-K In some embodiments, the respective control is () a directional control pad for a video game (e.g., soft-joystick, virtual d-pad/directional pad/joypad). For example, when the device detects contacts interacting with the directional control pad (and the contacts do not include the first fingerprint), the device performs corresponding operations associated with the directional control pad, such as navigating through a user interface or changing the direction of motion or view of a vehicle or character in a video game., for example, show devicedisplaying respective controlwhich is a directional control for the video game displayed on touch screen.

4212 100 4102 4102 112 4108 4112 4112 41 FIG.E 41 FIG.B 41 FIG.D b a In response to detecting the first fingerprint, the device displays () the respective control at the first location on the display., for example, shows devicedisplaying respective controlat first location-on touch screenin response to detecting first fingerprint(e.g., a user's right pinky finger) at the first location in, or first fingerprint(e.g., a user's right pinky finger) at first location-in.

4214 100 4102 4114 4102 4102 4102 100 4103 4101 4114 4102 41 FIG.G b In some embodiments, when the respective control is displayed at the first location, the device responds () to inputs corresponding to the first location by performing operations associated with the respective control. Additionally, in some embodiments, when the respective control is displayed at the first location, the device responds to inputs corresponding to the prior location by performing operations (e.g., scrolling, selecting, etc.) that are not associated with the respective control (or, optionally by not performing any operations if the inputs do not correspond to other operations in the user interface)., for example, shows deviceperforming operations associated with respective controlin response to detecting fingerprint(e.g., a user's right index finger) over respective controlwhile respective controlis displayed at first location-. In one example, devicechanges direction of an interactive object of the video game (e.g., vehicle) on user interfacein accordance with the location of fingerprintover respective controlcorresponding to a left direction.

4216 100 4102 112 4112 4112 41 FIG.D a. In some embodiments, the respective control is not displayed () on the display (immediately) prior to detecting the first fingerprint., for example, shows devicenot displaying respective controlon touch screenprior to detecting first fingerprint(e.g., a user's right pinky finger) at first location-

4218 100 112 100 4102 4102 4108 112 4112 4112 112 41 41 FIGS.A-K 41 FIG.E 41 FIG.B 41 FIG.D b a In some embodiments, the display is () a touch-screen display, and the first location at which the respective control is displayed corresponds to a location of the first fingerprint on the touch-screen display., for example, show portable multifunction devicewith touch screenwhich is a touch-sensitive display surface or a touch-screen display. Furthermore,, for example, shows devicedisplaying respective controlat first location-corresponding to the location of first fingerprint(e.g., a user's right pinky finger) at the first location inon touch screen, or first fingerprint(e.g., a user's right pinky finger) at first location-inon touch screen.

4220 100 4102 4102 112 100 4101 100 41 FIG.G b In some embodiments, the device displays () the respective control at the first location on the display in response to detecting the first fingerprint while the device is in a normal mode of operation that is not associated with reconfiguring the user interface of the device (e.g., the respective control first appears or is moved on the display while the device is not in a separate reconfiguration mode)., for example, shows devicedisplaying respective controlat first location-on touch screenwhile deviceis in a normal mode of operation (e.g., video game play mode) that is not associated with reconfiguring user interfaceof device.

4222 4224 100 4112 4112 100 4102 4102 112 4112 4112 41 FIG.G 41 FIG.E 41 FIG.G 41 FIG.E a b a In some embodiments, the device detects () liftoff of the first fingerprint, and after detecting liftoff of the first fingerprint, the device maintains () display of the respective control at the first location on the display. For example, the respective control is permanently or semi-permanently moved to the first location until the user moves the respective control again by placing the first fingerprint at another location on the touch-sensitive surface., for example, shows devicedetecting liftoff of first fingerprint(e.g., a user's right pinky finger) from first location-in., for example, further shows devicemaintaining display of respective controlat first location-on touch screenafter detecting liftoff of first fingerprintfrom first location-in.

4226 In some embodiments, while displaying the respective control at the first location on the display, the device detects () a respective fingerprint on the touch-sensitive surface that corresponds to a focus selector at a second location on the display, where the second location is different from the first location and does not include the respective control. In some embodiments, when the respective fingerprint is the first fingerprint, detecting the first fingerprint at the second location includes detecting a dragging gesture performed with the first fingerprint that corresponds to movement from the first location to the second location. In some embodiments, when the respective fingerprint is the first fingerprint, detecting the first fingerprint at the second location includes detecting a tap gesture performed with the first fingerprint at a location on the touch-sensitive surface that corresponds to the second location on the display.

41 FIG.F 41 FIG.H 41 FIG.D 41 FIG.I 41 FIG.D 100 4112 4112 4112 4102 4102 4102 4112 100 4116 4102 4112 4112 4102 4102 4116 100 4118 4102 4112 4112 4102 4102 4118 a b b a b a b , for example, shows devicedetecting a dragging gesture of respective fingerprintfrom first location-to second location-that does not include respective control, while displaying respective controlat first location-. In this example, respective fingerprintis the first fingerprint (e.g., a user's right pinky finger)., for example, shows devicedetecting respective fingerprint(e.g., a user's right middle finger) at a second location that does not include respective controland is different from first location-of first fingerprintin, while displaying respective controlat first location-. In this example, respective fingerprintis a second fingerprint (e.g., the user's right middle finger) different from the first fingerprint (e.g., a user's right pinky finger)., for example, shows devicedetecting respective fingerprint(e.g., a user's right pinky finger) at a second location that does not include respective controland is different from first location-of first fingerprintin, while displaying respective controlat first location-. In this example, respective fingerprintis the first fingerprint (e.g., a user's right pinky finger).

4228 4230 4232 100 4102 4102 4102 4102 100 4112 4112 4118 4112 4118 4102 41 FIG.J 41 FIG.F 41 FIG.I b c b In some embodiments, in response to detecting () the respective fingerprint and in accordance with a determination that the respective fingerprint is the first (predetermined) fingerprint associated with moving (e.g., relocating on the display) the respective control for the user interface on the touch-sensitive surface, the device: ceases () to display the respective control at the first location on the display; and displays () the respective control at the second location on the display (e.g., in response to detecting a subsequent input with the first fingerprint, the respective control is moved to a different location on the display)., for example, shows deviceceasing to display respective controlat first location-and displaying respective controlat second location-. In this example, deviceperforms the aforementioned operations in response to detecting respective fingerprintat second location-in, or respective fingerprintat the second location in, and in accordance with a determination that respective fingerprintoris the first fingerprint (e.g., a user's right pinky finger) associated with moving respective control.

4234 100 4102 4102 100 4116 4116 4102 41 FIG.K 41 FIG.H b In some embodiments, in response to detecting the respective fingerprint and in accordance with a determination that the respective fingerprint is a second fingerprint that is different from the first fingerprint and is not associated with moving (e.g., relocating on the display) the respective control for the user interface on the touch-sensitive surface, the device maintains () display of the respective control at the first location on the display. The device also, optionally, performs an operation associated with the second location on the display that is not an operation corresponding to the respective control (e.g., if the gesture performed with the second fingerprint is a gesture for interacting with an icon or other user interface element displayed at the second location on the display)., for example, shows devicemaintaining display of respective controlat first location-. In this example, deviceperforms the aforementioned operation in response to detecting respective fingerprintat second location inand in accordance with a determination that respective fingerprintis a second fingerprint (e.g., a user's right middle finger) different from the first fingerprint (e.g., a user's right pinky finger) and is not associated with moving respective control.

42 42 FIGS.A-C 42 42 FIGS.A-C 4200 4200 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, user interface objects, focus selectors described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, user interface objects, focus selectors described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

43 FIG. 43 FIG. 4300 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

43 FIG. 4300 4302 4304 4306 4308 4302 4304 4306 4308 4310 4312 4314 4316 As shown in, an electronic deviceincludes a display unitconfigured to display a graphic user interface, a touch-sensitive surface unitconfigured to receive one or more contacts, a fingerprint sensor unit; and a processing unitcoupled to the display unit, the touch-sensitive surface unit, and the fingerprint sensor unit. In some embodiments, the processing unitincludes a display enabling unit, a detecting unit, a performing unitand a determining unit.

4308 4312 4304 4302 4308 4310 4302 The processing unitis configured to detect (e.g., with the detecting unit) a first fingerprint associated with moving a respective control for the user interface on the touch-sensitive surface unit, where the first fingerprint corresponds to a focus selector at a first location on the display unitthat does not include the respective control. In response to detecting the first fingerprint, the processing unitis configured to enable display (e.g., with the display enabling unit) of the respective control at the first location on the display unit.

4302 In some embodiments, the respective control is not displayed on the display unitprior to detecting the first fingerprint.

4308 4310 4310 In some embodiments, the processing unitis configured to: enable display (e.g., with the display enabling unit) of the respective control at a prior location that is different from the first location, prior to detecting the first fingerprint; and in response to detecting the first fingerprint, cease display (e.g., with the display enabling unit) of the respective control at the prior location.

4308 4314 4314 In some embodiments, the processing unitis configured to: when the respective control is displayed at the prior location, respond to inputs corresponding to the prior location by performing (e.g., with the performing unit) operations associated with the respective control; and when the respective control is displayed at the first location, respond to inputs corresponding to the first location by performing (e.g., with the performing unit) operations associated with the respective control.

4302 In some embodiments, the display unitis a touch-screen display unit, and the first location at which the respective control is displayed corresponds to a location of the first fingerprint on the touch-screen display unit.

4308 4312 4310 4302 In some embodiments, the processing unitis further configured to: detect (e.g., with the detecting unit) liftoff of the first fingerprint; and after detecting liftoff of the first fingerprint, maintain display (e.g., with the display enabling unit) of the respective control at the first location on the display unit.

4308 4310 4302 4312 4304 4302 4316 4304 4310 4302 4310 4302 In some embodiments, processing unitis further configured to: while displaying (e.g., with the display enabling unit) the respective control at the first location on the display unit, detect (e.g., with the detecting unit) a respective fingerprint on the touch-sensitive surface unitthat corresponds to a focus selector at a second location on the display unit, where the second location is different from the first location and does not include the respective control; and in response to detecting the respective fingerprint, in accordance with a determination (e.g., with the determining unit) that the respective fingerprint is the first fingerprint associated with moving the respective control for the user interface on the touch-sensitive surface unit: cease to display (e.g., with the display enabling unit) the respective control at the first location on the display unit; and enable display (e.g., with the display enabling unit) of the respective control at the second location on the display unit.

4308 4310 4302 4316 4304 In some embodiments, the processing unitis further configured to maintain display (e.g., with the display enabling unit) of the respective control at the first location on the display unit, in response to detecting the respective fingerprint and in accordance with a determination (e.g., with the determining unit) that the respective fingerprint is a second fingerprint that is different from the first fingerprint and is not associated with moving the respective control for the user interface on the touch-sensitive surface unit.

In some embodiments, the respective control is a directional control pad for a video game.

4308 4310 4302 In some embodiments, the processing unitis configured to enable display (e.g., with the display enabling unit) of the respective control at the first location on the display unitin response to detecting the first fingerprint while the device is in a normal mode of operation that is not associated with reconfiguring the user interface of the device.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

42 42 FIGS.A-C 1 1 FIGS.A-B 43 FIG. 1 1 FIGS.A-B 4204 4212 4204 4222 4210 4214 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, displaying operationsand, detecting operationsand, and performing operationandare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices have graphical user interfaces that display content and allow the user to interact with the device. Sometimes, the user may want to expand the graphical user interface beyond a display integrated in, or connected to, the device itself. For example, a user operating a desktop or laptop computer with a built-in or external display optionally connects a second electronic device or external display to the CPU through a cable, and then configure the second display to expand the graphical user interface from the first display onto the second display. Likewise, a user operating a portable multifunction electronic device (e.g., a smart phone, tablet computer, or digital planner) optionally connects (e.g., via wired or wireless connection) the portable device to a base computer to more easily configure the device, transfer files onto or off of the portable device, etc. In this fashion, the effective size of the graphical user interface is increased, improving the user's ability to interact with the device. However, current methods for configuring a second electronic device as an auxiliary display of a first electronic device require opening menus and sub-menus and/or multiple gestures and button presses to configure the display of the second device, which can be confusing and/or time consuming for the user. Accordingly, there is a need to provide methods and user interfaces that enable to the user to more efficiently and conveniently configure a second electronic device as an auxiliary display for a first electronic device.

The embodiments described below provide improved methods and user interfaces for configuring a second electronic device as an auxiliary display of a first electronic device. More specifically, according to some embodiments described below, methods and user interfaces are provided that allow a user to configure a second device as an auxiliary display for a first device by merely touching a fingerprint sensor on each device. In some embodiments, the methods and user interfaces correlate a respective fingerprint identity from a touch on the first device to a respective fingerprint identity from a touch on the second device, and automatically configure the second device as an auxiliary display for the first device. Alternatively, in some embodiments, upon detection of the first and second fingerprints, the user is prompted to confirm the configuration of the second device as an auxiliary display for the first device. In some embodiments described below, the orientation of the touch on the first and/or second electronic device determines the orientation of content displayed on the auxiliary display. In some embodiments, the identity of the fingerprint touch on the first and/or second device determines the direction in which the user interface of the first display is expanded onto the auxiliary display. For example, in some embodiments, where the user touches the first device with their left hand and touches the second device with their right hand, the user interface is expanded to the right on the auxiliary display, and vise versa. Advantageously, the methods and user interfaces described below simplify the process of configuring a second electronic device as an auxiliary display of a first device by reducing the number of required steps.

450 451 100 112 167 44 44 45 45 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 450 451 359 2 359 1 359 1 450 112 169 359 1 359 2 112 112 4422 1 FIG.A 4 FIG.B 4 FIG.B 44 44 45 45 FIGS.A-EE andA-D 44 44 FIGS.A-EE 44 44 FIGS.A-EE 44 44 FIGS.A-EE 44 44 FIGS.A-EE In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toA-EE andA-D includes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated into the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, cither an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to display, a separate touch-sensitive surfaceand a separate fingerprint sensor-, however analogous operations are, optionally, performed on a device with an integrated fingerprint sensor-in response to detecting the inputs described inon the integrated fingerprint sensor-while displaying the user interfaces shown inon the display. Additionally, analogous operations are, optionally, performed on a device with a touch screenin response to detecting the contacts described inon a fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-) while displaying the user interfaces shown inon the touch screen; in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of cursor.

44 44 FIGS.A-J 100 1 100 2 112 100 1 100 2 169 100 1 100 2 112 169 illustrate two electronic devices (e.g., multifunction devices-and-), each having a touch screenwith a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, allowing them to be used as fingerprint sensors. Multifunction devices-and-are also illustrated as having separate fingerprint sensors. In some embodiments, each of multifunction devices-and-independently have a touch screenthat can be used as a fingerprint sensor, a separate fingerprint sensor, or both.

44 44 FIGS.A-F 100 1 4402 100 2 4403 4402 100 2 100 1 4403 4402 4403 100 1 100 2 100 2 100 1 4402 4403 100 2 100 1 illustrate various embodiments where first multifunction device-detects a first fingerprintand second multifunction device-detects a second fingerprintconcurrently with, or after detection of first fingerprint(e.g., the fingerprints are detected simultaneously, substantially simultaneously, or in close temporal proximity). Multifunction device-communicates to multifunction device-with information that second fingerprintwas detected. When predefined criteria are met by fingerprintsand, multifunction device-responds to multifunction device-with information that enables multifunction device-to be configured as an auxiliary display for multifunction device-. When predetermined criteria are not met by the detection of fingerprintsand, multifunction device-is not configured as an auxiliary display for multifunction device-.

44 44 FIGS.A-D 44 FIG.A 44 FIG.B 4402 4403 100 1 4404 1 100 2 4406 100 1 4402 100 2 4403 100 2 100 1 4403 illustrate an embodiment where the predefined criteria includes a criterion that the fingerprintsanddetected by both devices are registered to the same user. In, multifunction device-displays a first user interface that includes map-of the United States, while multifunction device-displays a second user interface including application launch screen. In, multifunction device-detects first fingerprint, corresponding to the left index (LI) finger of a user, and multifunction device-detects second fingerprint, corresponding to the right index (RI) finger of a user. Multifunction device-communicates to multifunction device-that second fingerprintwas detected and, optionally, additional information about the contact (e.g., the identity of the fingerprint).

44 FIG.C 4402 4403 4402 4403 112 100 1 100 2 4404 112 4404 1 112 1 100 1 4404 2 112 2 100 2 4402 4403 In, where fingerprintsandcorrespond to the same user (e.g., fingerprintis from the left index finger of user 1 and fingerprintis from the right index finger of user 1), the first user interface displayed on touch screenof multifunction device-is extended onto multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the western United States is displayed as map-on touch screen-of multifunction device-and the eastern United States is displayed as map-on touch screen-of multifunction device-), because the predefined criteria were satisfied (e.g., both fingerprintsandwere identified as belonging to the same user).

44 FIG.D 4405 4407 4405 4407 100 2 112 1 100 1 4404 1 112 2 100 2 4406 4405 4407 In, where fingerprintsandcorrespond to different users (e.g., fingerprintis from the left index finger of user 1 and fingerprintis from the right index finger of user 2), the first user interface is not extended onto multifunction device-(e.g., touch screen-of multifunction device-still displays the first user interface that includes map-and touch screen-of multifunction device-still displays application launch screen), because the predefined criteria were not satisfied (e.g., fingerprintsandwere identified as belonging to different users).

44 44 FIGS.E-F 44 44 FIGS.E andF 44 44 FIGS.E andF 44 FIG.E 44 FIG.F 4411 4415 4409 4413 100 1 4409 100 2 4411 4410 4408 100 2 100 1 4411 112 1 100 1 100 2 4404 112 4404 1 112 100 1 4404 2 112 2 100 2 4411 4409 100 1 4413 100 2 4415 4459 4408 100 2 100 1 4415 100 2 112 1 100 1 4404 1 112 2 100 2 4406 4415 4413 illustrate an embodiment where the predefined criteria includes a criterion that the second fingerprint (e.g., fingerprintsandin, respectively) is detected within a predetermined period of time after detection of the first fingerprint (e.g., fingerprintsandin, respectively), for example the second contact must occur within 5 seconds after the first contact is detected. For example, in, multifunction device-detects first fingerprintat a first time (e.g., defined as time 0) and multifunction device-detects second fingerprintat a second time, after the first time and prior to a predetermined time. Multifunction device-communicates to multifunction device-that second fingerprintwas detected and, optionally, additional information about the contact (e.g., the time at which it was detected). The first user interface displayed on touch screen-of multifunction device-is extended onto multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the western United States is displayed as map-on touch screenof multifunction device-and the eastern United States is displayed as map-on touch screen-of multifunction device-), because the predetermined criteria were satisfied (e.g., second fingerprintwas detected within five seconds of the detection of first fingerprint). In contrast, in, multifunction device-detects first fingerprintat a first time (e.g., defined as time 0) and multifunction device-detects second fingerprintat a second time, after the first time and after a predetermined time. Multifunction device-communicates to multifunction device-that second fingerprintwas detected and, optionally, additional information about the contact (e.g., the time at which it was detected). The first user interface is not extended onto multifunction device-(e.g., touch screen-of multifunction device-still displays the first user interface that includes map-and touch screen-of multifunction device-still displays application launch screen), because the predefined criteria were not satisfied (e.g., second fingerprintwas not detected within five seconds of the detection of first fingerprint).

44 44 FIGS.G-H 44 44 FIGS.G-H 44 FIG.G 44 FIG.H 44 FIG.G 44 FIG.H 44 44 FIGS.C andE 100 1 100 2 4417 4419 100 2 100 1 4419 4417 4419 100 2 100 1 4412 112 2 100 2 4421 112 1 100 1 100 2 112 100 1 100 2 4404 112 4404 1 112 1 100 1 4404 2 112 2 100 2 illustrate various embodiment where user confirmation of the request to configure the second device as an auxiliary display is required. In, multifunction device-and multifunction device-detect fingerprintand, respectively. Multifunction device-communicates to multifunction device-with information that second fingerprintwas detected. When predefined criteria are otherwise met by the detection of fingerprintsand, the user of second multifunction device-(as illustrated in) and/or the user of first multifunction device-(as illustrated in) is prompted (e.g., by display of prompton touch screen-of multifunction device-inor by display of prompton touch screen-on multifunction device-in) to confirm that second multifunction device-should be configured as an auxiliary display (e.g., the user has to verify that a request to connect to Abraham's and/or Mary Todd's iPhone has been made). Upon receiving confirmation of the request, the first user interface displayed on touch screenof multifunction device-is extended onto multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the western United States is displayed as map-on touch screen-of multifunction device-and the eastern United States is displayed as map-on touch screen-of multifunction device-, as illustrated in).

441 44 FIGS.-J 44 FIG.I 44 FIG.J 4427 4429 100 1 4427 100 2 4429 100 2 100 1 4429 4427 4429 112 1 100 1 112 2 100 2 4429 4404 112 4404 1 112 1 100 1 4404 2 112 2 100 2 illustrate an embodiment where an orientation of a fingerprint (e.g., fingerprintand/or fingerprint) determines the orientation of the user display on the auxiliary display. In, multifunction device-detects first fingerprint, having a first orientation substantially parallel with the long axis of the device. Multifunction device-detects second fingerprint, having a second orientation substantially perpendicular to the long axis of the device. Multifunction device-communicates to multifunction device-with information that second contactwas detected and, optionally, additional information about the contact (e.g., the orientation of the contact). When predefined criteria are otherwise met by detection of fingerprintsand, the first user interface displayed on touch screen-of multifunction device-is extended onto touch screen-of multifunction device-in a landscape orientation corresponding to the orientation of second fingerprintrelative to the long axis of the device, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the western United States is displayed as map-on touch screen-of multifunction device-and the eastern United States is displayed as map-on touch screen-of multifunction device-), as illustrated in.

44 44 FIGS.K-P 44 44 FIGS.K-P 100 1 100 2 100 3 169 illustrate three electronic devices (e.g., multifunction devices-,-, and-), each having a fingerprint sensor. Various embodiments are illustrated inwhere the identity of a fingerprint determines the direction in which a user interface displayed on a first electronic device is expanded onto the second electronic device.

44 44 FIGS.K-N 44 44 FIGS.K-L 44 44 FIGS.M-M 44 FIG.K 44 FIG.L 44 FIG.M 44 FIG.N 4437 4441 4435 169 1 100 1 4437 169 2 100 2 4435 4437 112 1 100 1 112 2 100 2 4404 112 4404 1 112 1 100 1 4404 2 112 2 100 2 4439 169 1 100 1 4441 169 3 100 3 4439 4441 112 1 100 1 112 3 100 3 4404 112 4404 1 112 1 100 1 4404 3 112 3 100 3 In some embodiments, as illustrated in, the hand from which the finger corresponding to second fingerprint (e.g., fingerprintinor fingerprintin) belongs determines the direction in which the user interface is expanded with the auxiliary display. For example, in, fingerprint, corresponding to the user's left index (LI) finger, is detected by fingerprint sensor-on first multifunction device-. Fingerprint, corresponding to the user's right index (RI) finger, is detected by fingerprint sensor-on second multifunction device-. In response, as illustrated in, when predefined criteria are otherwise met by the detection of fingerprintsand, the first user interface displayed on touch screen-of multifunction device-is extended to the right onto touch screen-of multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the central United States is displayed as map-on touch screen-of multifunction device-and the eastern seaboard of the United States is displayed as map-on touch screen-of multifunction device-), because the right index finger is located on the user's right hand. Similarly, as illustrated in, fingerprint, corresponding to the user's right index (RI) finger, is detected by fingerprint sensor-on first multifunction device-. Fingerprint, corresponding to the user's left index (LI) finger, is detected by fingerprint sensor-on second multifunction device-. In response, as illustrated in, when predefined criteria are otherwise met by the detection of fingerprintsand, the first user interface displayed on touch screen-of multifunction device-is extended to the left onto touch screen-of multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the central United States is displayed as map-on touch screen-of multifunction device-and the west coast of the United States is displayed as map-on touch screen-of multifunction device-), because the left index finger is located on the user's left hand.

44 44 FIGS.O-P 44 FIG.O 44 FIG.P 4444 4443 4443 169 1 100 1 4444 169 2 100 2 4443 4444 112 1 100 1 112 2 100 2 4404 112 4404 1 112 1 100 1 4404 2 112 2 100 2 In some embodiments, as illustrated in, the order on the user's hands, of the finger corresponding to the second fingerprint (e.g., fingerprint) relative to the finger corresponding to the first fingerprint (e.g., fingerprint) determines the direction in which the user interface is expanded with the auxiliary display. For example, in, fingerprint, corresponding to the user's right ring (RR) finger, is detected by fingerprint sensor-on first multifunction device-. Fingerprint, corresponding to the user's right index (RI) finger, is detected by fingerprint sensor-on second multifunction device-. In response, as illustrated in, when predefined criteria are otherwise met by the detection of fingerprintsand, the first user interface displayed on touch screen-of multifunction device-is extended to the left onto touch screen-of multifunction device-, such that mapof the United States is displayed on touch screensof both multifunction devices (e.g., the central United States is displayed as map-on touch screen-of multifunction device-and the west coast of the United States is displayed as map-on touch screen-of multifunction device-), because the right index finger is located to the user's left of the right ring finger on the user's hands.

44 44 FIGS.Q-EE 44 44 FIGS.Q-T 100 1 100 2 112 100 1 100 2 169 100 1 100 2 112 169 100 2 100 1 100 1 illustrate two electronic devices (e.g., multifunction devices-and-), each having a touch screenwith a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, allowing them to be used as fingerprint sensors. Multifunction devices-and-are also illustrated as having separate fingerprint sensors. In some embodiments, each of multifunction devices-and-independently have a touch screenthat can be used as a fingerprint sensor, a separate fingerprint sensor, or both. In, various embodiments are illustrated where second multifunction device-, while configured as an auxiliary display for first multifunction device-, displays a set of user interface elements associated with an application that is active on first multifunction device-.

44 FIG.Q 44 FIG.R 100 1 4404 1 100 2 4406 4445 4446 100 2 100 1 112 2 100 2 4414 4416 4447 100 1 4414 4447 4416 In, multifunction device-displays a first user interface that includes map-of the United States, corresponding to an active photo display application. Multifunction device-displays a second user interface including application launch screen. Upon detection of qualifying fingerprintsand(e.g., fingerprints that meet the predefined auxiliary-display criteria), second multifunction device-is configured as an auxiliary display for first multifunction device-. As illustrated in, while configured as an auxiliary display, touch screen-of multifunction device-displays user interface elements,, andthat are associated with the photo display application active on first multifunction device-(e.g., toolbarsandand drop-down menu).

44 44 FIGS.S-T 44 FIG.S 44 FIG.T 100 2 100 1 4430 112 2 4414 100 1 4430 100 2 112 1 100 1 4404 1 4432 4432 1 4432 2 4432 3 4432 4 4430 100 2 112 2 100 2 100 1 4448 4432 112 1 100 1 b In some embodiments, as illustrated in, the user interface elements displayed on the auxiliary display are controls for performing operations in the application active on the first display. In, second multifunction device-, configured as an auxiliary display for first multifunction device-, detects contactat a position on touch screen-corresponding to a control for displaying photo albums (e.g., user interface menu element-) in the photo display application active on multifunction device-. In response to detecting contacton second multifunction device-, the user interface displayed on touch screen-of first multifunction device-is changed from displaying map-of the United States to displaying launch iconsfor photo albums (e.g., icons representing photo albums “Flowers”-, “Outer Space”-, “Vacation”-, and “Winter”-), as illustrated in(e.g., the mode of the photo display application is changed from displaying locations where pictures were taken to displaying photo albums compiled by a user). Additionally, in response to detecting contacton second multifunction device-, touch screen-of multifunction device-, configured to as an auxiliary display for multifunction device-, displays drop-down menucorresponding to controls for performing functions relating to the photo albums represented by launch iconsdisplayed on touch screen-of multifunction device-.

44 44 FIGS.U-V 44 FIG.U 44 FIG.V 44 FIG.U 44 FIG.V 100 1 4417 1 100 2 4406 4449 4450 100 2 100 1 4417 112 100 1 100 2 100 2 4418 1 4418 112 1 100 1 1 4417 2 4417 112 2 100 2 In some embodiments, as illustrated in, the user interface elements displayed on the auxiliary display include a portion of a workspace that was expanded from the display of the first device. In, multifunction device-displays a first user interface that includes workspace-of an active word processing application. Multifunction device-displays a second user interface including application launch screen. Upon detection of qualifying fingerprintsand(e.g., fingerprints that meet the predefined auxiliary-display criteria), second multifunction device-is configured as an auxiliary display for first multifunction device-. As illustrated in, the first user interface is expanded with the auxiliary display to create a continuous workspacespanning touch screensof multifunction devices-and-. For example, when second multifunction device-is configured as an auxiliary display, the word “ago” (e.g., user interface object-), displayed on the second line of texton touch screen-of multifunction device-in, moves onto lineof workspace-(e.g., the portion of continuous workspacedisplayed on the auxiliary display) on touch screen-of second multifunction device-in.

44 44 FIGS.W-Y 44 FIG.W 44 FIG.X 44 FIG.X 44 FIG.Y 44 FIG.X 44 FIG.Y 100 1 4404 1 100 2 4406 4451 4452 100 2 100 1 112 2 100 2 4420 4453 2 112 1 100 1 4404 1 100 1 100 2 100 1 100 2 4426 4424 4422 112 1 100 1 4424 4424 100 1 4422 4422 4422 a b a b In some embodiments, as illustrated in, while the second electronic device is configured as an auxiliary display for the first electronic device, inputs corresponding to interactions with user interface elements are received by the second device and operations associated with an active application are performed on the first device. For example, in, multifunction device-displays a first user interface that includes map-of the United States, corresponding to an active photo display application. Multifunction device-displays a second user interface including application launch screen. Upon detection of qualifying fingerprintsand(e.g., fingerprints that meet the predefined auxiliary-display criteria), second multifunction device-is configured as an auxiliary display for first multifunction device-. As illustrated in, while configured as an auxiliary display, touch screen-of second multifunction device-displays user interface, which includes the same view of the United States (e.g., as map-) as displayed on touch screen-of first multifunction device-(e.g., instead of extending the display of map-on first multifunction device-, second multifunction device-mirrors the display of first multifunction device-). Second multifunction device-detects a gesture including movementof contact, corresponding to focus selectordisplayed on touch screen-of first multifunction device-, from position-into position-in. In response, first multifunction device-displays movement of focus selectorfrom position-into position-in.

44 44 FIGS.Z-BB 44 FIG.Z 44 FIG.AA 44 FIG.BB 100 1 4428 1 100 2 4406 4455 4456 100 2 100 1 112 2 100 2 4428 4428 2 100 2 4430 112 2 4457 100 2 4430 4457 100 1 100 2 4457 4429 100 2 100 1 100 2 4457 In some embodiments, as illustrated in, while the second electronic device is configured as an auxiliary display for the first electronic device, operational data from the second device, specifying operations corresponding to interactions with user interface elements on the auxiliary display associated with an active application on the first device, is received and performed by the first device. For example, in, multifunction device-displays a first user interface that includes keyboard-, corresponding to an active piano application. Multifunction device-displays a second user interface including application launch screen. Upon detection of qualifying fingerprintsand(e.g., fingerprints that meet the predefined auxiliary-display criteria), second multifunction device-is configured as an auxiliary display for first multifunction device-. As illustrated in, while configured as an auxiliary display, touch screen-of second multifunction device-displays an extension of keyboard(e.g., keyboard-, corresponding to a second octave of tones on the keyboard). In, second multifunction device-detects contactat a position on touch screen-corresponding to the display of piano key(e.g., the “E” key). Second multifunction device-processes contactto determine that an operation including generating sound at a pitch corresponding to “E” keyshould be performed. First multifunction device-receives a communication from second multifunction device-that includes operational data relating to generating sound at the pitch corresponding to “E” key, and generates soundaccording to the operational data received from second multifunction device-. In some embodiments, instead of sending the communication to First multifunction device-, Second multifunction device-generates a sound at the pitch corresponding to “E” key.

44 44 FIGS.CC-EE 44 FIG.CC 44 FIG.DD 44 FIG.DD 44 FIG.EE 44 FIG.EE 44 FIG.DD 44 FIG.EE 112 100 1 4432 4432 1 4432 2 4432 3 4432 4 100 2 4406 4460 4462 100 2 100 1 4432 4432 5 4432 6 112 2 100 2 100 1 4438 4436 112 1 4436 4432 4 4436 100 2 4442 4440 112 2 4440 4440 4432 4 4436 112 1 100 1 4440 112 2 100 2 a b a b a b In some embodiments, as illustrated in, while the second electronic device is configured as an auxiliary display for the first electronic device, corresponding gestures performed on touch-sensitive surfaces (e.g., touch screens) of each respective device causes a user interface object associated with an active application on the first device to move between the display of the first device and the display of the second device. For example, in, multifunction device-displays a first user interface that includes launch iconsfor photo albums (e.g., icons representing photo albums “Flowers”-, “Outer Space”-, “Vacation”-, and “Winter”-), corresponding to an active photo display application. Multifunction device-displays a second user interface including application launch screen. Upon detection of qualifying fingerprintsand(e.g., fingerprints that meet the predefined auxiliary-display criteria), second multifunction device-is configured as an auxiliary display for first multifunction device-. As illustrated in, the first user interface is expanded with the auxiliary display to display launch iconsfor additional photo albums (e.g., icons representing photo albums “Desert”-and “Wedding”-) on touch screen-of second multifunction device-. First multifunction device-detects a first gesture including movementof contacton touch screen-from position-in, corresponding to the display of launch icon-, to position-in. Second multifunction device-detects a second gesture including movementof contacton touch screen-from position-to position-in. In response, the display of launch icon-is moved from position-on touch screen-of first multifunction device-into position-on touch screen-of second multifunction device-in.

45 45 FIGS.A-D 3 FIG. 1 FIG.A 4500 4500 300 100 4500 are flow diagrams illustrating a methodof configuring a second electronic device as an auxiliary display of a first electronic device in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

4500 As described below, the methodprovides an intuitive way to configure a second electronic device as an auxiliary display of a first electronic device. The method reduces the cognitive burden on a user when configuring a second electronic device as an auxiliary display of a first electronic device, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to configure a second electronic device as an auxiliary display of a first electronic device faster and more efficiently conserves power and increases the time between battery charges.

4502 100 1 4402 4405 4409 4413 4417 4423 4427 4431 4435 4439 4443 4445 4449 4451 4455 4460 169 1 112 1 44 44 44 44 44 44 44 44 44 44 FIGS.B,D-K,M,O,Q,U,W,Z, andCC In some embodiments, a first electronic device with a fingerprint sensor and a display detects () a first fingerprint on a fingerprint sensor of a first electronic device at a first time, where the first fingerprint is registered to a respective user. For example,illustrate various embodiments where multifunction device-detects a first fingerprint (e.g., fingerprint,,,,,,,,,,,,,,, or) on fingerprint sensor-or touch screen-having a spatial resolution that is high enough to detect fingerprint features formed by individual fingerprint ridges, allowing it to be used as a fingerprint sensor.

4504 100 2 100 3 4403 4407 4411 4415 4419 4425 4429 4433 4437 4441 4444 4446 4450 4452 4456 4462 169 2 169 3 112 2 112 3 100 1 44 44 44 44 44 44 44 44 44 44 FIGS.B,D-K,M,O,Q,U,W,Z, andCC The first electronic device receives () a communication from a second electronic device, distinct from the first electronic device, that includes information indicating that a second fingerprint was detected on a fingerprint sensor of the second device at a second time. For example,illustrate various embodiments where multifunction device-or-detects a second fingerprint (e.g., fingerprint,,,,,,,,,,,,,,, or) on fingerprint sensor-or-, or on touch screen-or-and communicates information to multifunction device-that the second fingerprint was detected and, optionally, additional characterizing information about the contact (e.g., an identity of the fingerprint, a time the fingerprint was detected, or an orientation of the fingerprint).

4506 4508 4402 4403 4405 4407 44 FIG.C 44 FIG.D In response () to detecting the first fingerprint and receiving the communication from the second electronic device: the first electronic device determines () whether predefined auxiliary-display criteria have been met, where the auxiliary-display criteria include a criterion that is met when the second fingerprint is registered to the respective user. For example, as illustrated in, because fingerprintsandare registered to the same user (e.g., user “1”), the auxiliary-display criterion is satisfied. In contrast, as illustrated in, because fingerprintsandare registered to different users (e.g., users “1” and “2,” respectively), the auxiliary-display criterion is not satisfied.

4510 4411 4410 4409 4408 4409 4415 4459 4413 4408 4413 44 FIG.E 44 FIG.F In some embodiments, the auxiliary-display criteria include () a criterion that is met when the first time is within a predefined time of the second time. For example, the second device is used as an auxiliary display when the first fingerprint and the second fingerprint are detected simultaneously or substantially simultaneously on the first and second devices (e.g., within 0.5, 1.0, 2, or 5 seconds of each other). For example, as illustrated in, because second fingerprintwas detected at a time(e.g., three and a half seconds after detection of first fingerprint) within predetermined time(e.g., five seconds after detection of first fingerprint), the auxiliary-display criterion is satisfied. In contrast, as illustrated in, because second fingerprintoccurred at a time(e.g., six and a half seconds after detection of first fingerprint) not within predetermined time(e.g., five seconds after detection of first fingerprint), the auxiliary-display criterion is not satisfied.

4512 100 2 4412 4421 100 1 100 2 100 2 4402 4403 44 44 FIGS.G-H 44 44 FIGS.G andH 44 44 FIGS.A-C In some embodiments, the auxiliary-display criteria include () a criterion that is met when the first device receives a confirmation that configuration of the second device as an auxiliary display has been requested. In some embodiments, as illustrated in, configuration of the second device (e.g., multifunction device-) as the auxiliary display includes selecting “approve connection,” “approve,” “connect,” or a similar affirmative selection on one or both devices (e.g., the “CONNECT” prompt displayed in user interface objectsandin, respectively) and/or entering a predefined pin or password on one or both of the devices (e.g., multifunction device-and/or multifunction device-). In contrast, in some embodiments, as illustrated in the series of, the second device (e.g., multifunction device-) is configured as an auxiliary display for the first device without requesting any additional authentication performed prior to configuring the second device as an auxiliary device for the first device (e.g., the configuration happens automatically when the two fingerprintsandare detected).

4514 100 1 100 2 100 3 100 1 44 44 44 44 44 44 44 44 44 44 44 FIGS.C,E,J,L,N,P,R,V,X,AA, andDD In accordance with a determination that the auxiliary-display criteria are met, the first electronic device sends () a response to the second electronic device that enables the second electronic device to be configured as an auxiliary display for the first electronic device. For example,illustrate various embodiments where, in response to meeting the auxiliary-display criteria, first multifunction device-sent a response that enabled second multifunction device-or-to become configured as an auxiliary display for first multifunction device-.

4516 4427 100 1 4404 1 112 1 100 1 4433 100 2 4404 2 112 2 100 2 441 44 FIGS.-J 44 FIG.I 44 FIG.J 44 FIG.I 44 FIG.J In some embodiments, an orientation of a user interface displayed on the display of the first device is determined () in accordance with the orientation of the first fingerprint relative to a predefined axis of the display of the first device, and an orientation of a user interface displayed on the display of the second device is determined in accordance with the orientation of the second fingerprint relative to a predefined axis of the display of the second device. For example, the user interface displayed on the first device and the user interface displayed on the second device are oriented so that they are each aligned with the fingerprint detected on the corresponding devices (e.g., the user interfaces are oriented so that an “up” of the user interface on the displays corresponds to the “up” of the corresponding fingerprint on the fingerprint sensors). Thus, in some embodiments, as illustrated in, when a vertical axis of the fingerprint is substantially parallel to a predefined vertical axis of a respective device (e.g., where vertical axis of first contactis substantially parallel to the long axis of first multifunction device-in), then the user interface is aligned so that the bottom of the user interface is along a bottom edge of the respective device, with respect to the predefined vertical axis (e.g., the user interface displaying map-of the United States on touch screen-of first multifunction device-is displayed in a portrait orientation in), whereas, when a vertical axis of the fingerprint is substantially perpendicular to the predefined vertical axis of a respective device (e.g., where vertical axis of second contactis substantially perpendicular to the long axis of second multifunction device-in), then the user interface is aligned so that the bottom of the user interface is along a side edge of the respective device, with respect to the predefined vertical axis (e.g., the user interface displaying map-of the United States on touch screen-of second multifunction device-is displayed in a landscape orientation in).

4518 4439 100 1 4441 100 3 112 1 100 1 100 3 4404 3 112 3 100 3 4404 1 112 1 100 1 4435 100 1 4437 100 2 112 1 100 1 100 2 4404 2 112 2 100 2 4401 1 112 1 100 1 44 FIG.M 44 FIG.M 44 FIG.M 44 FIG.N 44 FIG.K 44 FIG.K 44 FIG.K 44 FIG.L In some embodiments, the first electronic device determines () a direction to extend a user interface displayed on the display of the first device with the auxiliary display in accordance with at least one of an identity of the first fingerprint and an identity of the second fingerprint. For example, when a fingerprint of a right hand of a user is detected on the fingerprint sensor of the first device (e.g., fingerprinton first multifunction device-corresponds to the user's right index finger in) and/or a fingerprint of a left hand of the user is detected on the fingerprint sensor of the second device (e.g., fingerprinton second multifunction device-corresponds to the user's left index finger in), the user interface of the first device is extended to the left onto the display of the second device (e.g., the first user interface displayed on touch screen-of multifunction device-inis extended onto multifunction device-in, such that the west coast of the United States is displayed as map-on touch screen-of second multifunction device-and the central United States is displayed as map-on touch screen-of first multifunction device-). Similarly, when a fingerprint of the left hand of the user is detected on the fingerprint sensor of the first device (e.g., fingerprinton first multifunction device-corresponds to the user's left index finger in) and/or a fingerprint of the right hand of the user is detected on the fingerprint sensor of the second device (e.g., fingerprinton second multifunction device-corresponds to the user's right index finger in), the user interface of the first device is extended to the right onto the display of the second device (e.g., the first user interface displayed on touch screen-of multifunction device-inis extended onto multifunction device-in, such that the eastern seaboard of the United States is displayed as map-on touch screen-of second multifunction device-and the central United States is displayed as map-on touch screen-of first multifunction device-). Thus, in some embodiments, the device that detects fingerprints from the left hand of the user is assumed to be to the left of the device that detects fingerprints from the right hand of the user, so user interface objects that are supposed to be on the left side of the user interface are displayed on the device that detected fingerprints from the user's left hand while user interface objects that are supposed to be on the right side of the user interface are displayed on the device that detected fingerprints from the user's right hand.

4444 100 2 4443 100 1 112 1 100 1 100 2 4404 2 112 2 100 2 4404 1 112 1 100 1 44 FIG.O 44 FIG.O 44 FIG.O 44 FIG.P In some embodiments, the order on the user's hands of the finger corresponding to the second fingerprint relative to the finger corresponding to the first fingerprint determines the direction in which the user interface is expanded with the auxiliary display. For example, when a second fingerprint (e.g., fingerprintdetected on second multifunction device-in) corresponding to a second finger (e.g., the user's right index (RI) finger) located to the user's left of a first finger (e.g., the user's right ring (RR) finger) corresponding to a first fingerprint (e.g., fingerprintdetected on first multifunction device-in), the user interface of the first device is extended to the left onto the display of the second device (e.g., the first user interface displayed on touch screen-of multifunction device-inis extended onto multifunction device-in, such that the west coast of the United States is displayed as map-on touch screen-of second multifunction device-and the central United States is displayed as map-on touch screen-of first multifunction device-).

4420 100 1 4522 100 2 100 1 44 44 44 FIGS.Q,W, andCC 44 FIG.U 44 FIG.Z 44 44 44 44 44 FIGS.R,V,X,AA, andDD In some embodiments, where the first fingerprint and the second fingerprint are detected () while a first application is an active application at the first device (e.g., where a photo display application is active as shown in; where a word processing application is active as shown in; and where a piano application is active as shown in, at first multifunction device-), and after sending a response to the second electronic device that enables the second electronic device to be configured as an auxiliary display for the first electronic device: the first device configures () the second electronic device as an auxiliary display for the first device (e.g., second multifunction device-is configured as an auxiliary display for first multi function device-in).

4524 4414 4447 4416 4418 4404 2 4428 2 4432 5 4432 6 44 FIG.R 44 FIG.V 44 FIG.X 44 FIG.AA 44 FIG.DD While the second electronic device is configured as an auxiliary display for the first device, the first device sends () instructions to the second device that enable the second device to display a set of one or more user interface elements associated with the first application (e.g., menu barsand, and drop-down menuin; textin; map-of the United States in; piano keyboard-in; and icons representing photo albums “Desert”-and “Wedding”-in).

4526 4414 4447 4416 44 FIG.R In some embodiments, the set of one or more user interface elements associated with the first application include () one or more controls for performing operations in the first application (e.g., the one or more user interface elements include: a menu, control element, or other user interface object associated with the first application). For example, using the display of the second device as an auxiliary display includes placing one or more of the controls for the user interface displayed on the display of the first device on the display of the second device (e.g., menu barsand, and drop-down menuin). Thus, additional menus, palettes or other controls can be displayed on the auxiliary display to reduce or eliminate cluttering the primary display (e.g., the user interface of the first application that is displayed on the display of the first device).

4528 4417 1 112 1 100 1 112 2 100 2 4417 2 4417 44 FIG.U 44 FIG.V In some embodiments, the set of one or more user interface elements associated with the first application include () a portion of a workspace that is a continuation of a workspace displayed on the display of the first device (e.g., the one or more user interface elements include: a document body, application canvas or other virtual workspace that is stretched over the displays of the first and second devices). For example, using the display of the second device as an auxiliary display includes spreading a workspace for the first application over the display of the first device and the display of the second device (e.g., workspace-is expanded from touch screen-of first multifunction device-inonto touch screen-of second multifunction device-, as workspace-of continuous workspacein).

4530 4532 4424 4426 4424 4424 4424 100 1 4422 4422 4422 100 2 44 FIG.X 44 FIG.X 44 FIG.Y 44 FIG.X 44 FIG.Y a b a b In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the first device receives () input data indicative of inputs detected by the second device, where the inputs correspond to interactions with one or more of the user interface elements associated with the first application that are displayed on the display of the second device; and the first device performs () operations associated with the first application in accordance with the input data. For example, the first device sends display information to the second device and receives input data (e.g., coordinates of touch points such as contactinand/or a gesture including movementof contactfrom position-into position-in) indicative of the user's interaction with the user interface displayed on the display of the second device; and in response the first device performs an operation associated with an active program in accordance with the first application (e.g., first multifunction device-moves focus selectorfrom position-into position-inin accordance with the gesture data provided by second multifunction device-). Thus, in some embodiments, the second device acts as an auxiliary touchpad or touch screen for the first device.

4534 4536 4428 4428 1 4428 2 100 1 100 2 4430 100 2 4457 100 2 4430 4457 100 1 44 FIG.BB 44 44 FIGS.AA-BB 44 FIG.BB 44 FIG.BB 44 FIG.BB In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the first device receives () operation data from the second electronic device that specifies one or more operations corresponding to interactions with one or more of the user interface elements associated with the first application that are displayed on the display of the second device; and performs () the one or more operations. Thus, in some embodiments, some application processing is done at each of the devices (e.g., the first device and the second device each have copies of the same application), the two applications each produce responses to gestures appropriate to the portion of the user interface that is displayed, and the two applications share information. For example, for a musical keyboard (e.g., piano keyboardin) that is spread across the display of the first device and the display of the second device (e.g., as keyboards-and-displayed on multifunction devices-and-, respectively, in), instead of sending touch points to the first electronic device (e.g., coordinates of contactin), the second device (e.g., multifunction device-in) displays half of the keys of the musical keyboard (e.g., including “E” keyin) and sends information to the first device indicating a note to play corresponding to a key that was activated by the user (e.g., second multifunction device-determines that contactshould activate “E” keyand sends information to first multifunction device-to play a sound having pitch “E”). As another example, for a split text keyboard, instead of sending touch points to the first electronic device, the second electronic device sends letters that correspond to the keys that were activated by the user. By performing some of the processing at the second device, the processing load on the first device is decreased, thereby improving the performance of the application.

4538 4540 4542 100 1 4438 4436 4436 4436 112 1 4436 4432 4 100 2 4442 4440 4440 4440 112 2 4440 4432 4 4432 4 44 FIG.DD 44 FIG.DD 44 FIG.EE 44 FIG.DD 44 FIG.DD 44 FIG.EE 44 FIG.EE 44 FIG.EE a b a a b b In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the first device detects () a first gesture on a touch-sensitive surface of the first device (e.g., the first gesture corresponds to a first location on a touch-sensitive display of the first device), and receives () a communication from the second device indicative of a second gesture detected on a touch-sensitive surface of the second device (e.g., the second gesture corresponds to a second location on a touch-sensitive display of the second device). In response to detecting the first gesture and receiving the communication indicative of the second gesture, the first device moves () a user interface object (e.g., a file, icon, menu, or control) associated with the first application between the display of the first device (e.g., a first location on the display of the first device) and the display of the second device (e.g., a second location on the display of the second device). For example, the first device detects a pinch gesture on the touch-sensitive surface of the first device at a first location that corresponds to a first user interface object and the second device detects a depinch gesture on the touch-sensitive surface of the second device at a second location, and in response to detecting the pinch and depinch gestures, the first user interface object is moved from the first location to the second location (e.g., the first device ceases to display the first user interface object and the second device starts to display the first user interface object). As another example, the first device (e.g., first multifunction device-in) detects a first portion of a drag gesture on the first touch-sensitive surface (e.g., a drag gesture including movementof contactfrom position-into position-on touch screen-in) that starts at a first location (e.g., position-in) that corresponds to a first user interface object (e.g., icon-representing photo album “Winter” in) and the second device (e.g., multifunction device-) detects a second portion of the drag gesture on the second touch-sensitive surface (e.g., a drag gesture including movementof contactfrom position-to position-on touch screen-in) that ends (e.g., with liftoff of the contact) at a second location (e.g., position-in), and in response to the first and second portions of the drag gesture, the first user interface object is moved from the first location to the second location (e.g., the first device ceases to display icon-representing photo album “Winter” and the second device starts to display icon-representing photo album “Winter,” as illustrated in).

4544 4405 4407 4405 4407 100 2 100 1 4415 4408 4413 4415 4413 100 2 100 1 44 FIG.D 44 FIG.F In accordance with a determination that the auxiliary-display criteria are not met, the first device foregoes () sending the response to the second electronic device that enables the second electronic device to be configured as the auxiliary display for the first electronic device. For example, in, the auxiliary-display criteria are not met because first fingerprintand second fingerprintare not registered to the same user (e.g., fingerprintcorresponds to user “1” and fingerprintcorresponds to user “2”), and thus, second multifunction device-is not configured as an auxiliary display for first multifunction device-. Similarly, in, the auxiliary-display criteria are not met because second fingerprintwas not detected within predetermined timeafter detection of first fingerprint(e.g., fingerprintwas detected more than 5 seconds after fingerprintwas detected), and thus, second multifunction device-is not configured as an auxiliary display for first multifunction device-.

45 45 FIGS.A-D 45 45 FIGS.A-D 4500 4500 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints, contacts, gestures, user interface objects, and focus selectors described above with reference to methodoptionally have one or more of the characteristics of the fingerprints, contacts, gestures, user interface objects, and focus selectors described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

46 FIG. 46 FIG. 4600 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

46 FIG. 4600 4602 4604 4606 4608 4602 4604 4606 4608 4610 4612 4614 4616 4618 4602 4620 4622 4624 As shown in, an electronic deviceincludes a display unitconfigured to display a graphic user interface, optionally, a touch-sensitive surface unit, a fingerprint sensor unit; and a processing unitcoupled to the display unit, touch-sensitive surface unit, and the fingerprint sensor unit. In some embodiments, the processing unitincludes a detecting unit, a receiving unit, a determining unit, a sending unit, a display enabling unitfor enabling display of information on display unit, a configuring unit, a performing unit, and a moving unit.

4608 4600 4610 4608 4612 4600 4608 4614 4608 4616 4600 4608 4616 4600 The processing unitis configured to detect a first fingerprint on the fingerprint sensor unit of the first electronic deviceat a first time (e.g., with the detecting unit), where the first fingerprint is registered to a respective user. The processing unitis also configured to receive a communication from a second electronic device (e.g., with the receiving unit), distinct from the first electronic device, that includes information indicating that a second fingerprint was detected on a fingerprint sensor unit of the second device at a second time. In response to detecting the first fingerprint and receiving the communication from the second electronic device, the processing unitis configured to determine whether predefined auxiliary-display criteria have been met (e.g., with the determining unit), where the auxiliary-display criteria include a criterion that is met when the second fingerprint is registered to the respective user. In accordance with a determination that the auxiliary-display criteria are met, the processing unitis configured to send a response to the second electronic device (e.g., with the sending unit) that enables the second electronic device to be configured as an auxiliary display for the first electronic device. In accordance with a determination that the auxiliary display criteria are not met, the processing unitis configured to forego sending the response to the second electronic device (e.g., with the sending unit) that enables the second electronic device to be configured as the auxiliary display for the first electronic device.

In some embodiments, the auxiliary-display criteria include a criterion that is met when the first time is within a predefined time of the second time.

4612 In some embodiments, the auxiliary-display criteria include a criterion that is met when the first device receives a confirmation that configuration of the second device as an auxiliary display has been requested (e.g., with the receiving unit).

4602 4600 4602 4600 4614 In some embodiments, an orientation of a user interface displayed on the display unitof the first deviceis determined in accordance with the orientation of the first fingerprint relative to a predefined axis of the display unitof the first device(e.g., with the determining unit), and an orientation of a user interface displayed on the display unit of the second device is determined in accordance with the orientation of the second fingerprint relative to a predefined axis of the display unit of the second device (e.g., with the determining unit).

4608 4600 4614 In some embodiments, the processing unitis further configured to determine a direction to extend the user interface of the first devicewith the auxiliary display in accordance with at least one of an identity of the first fingerprint and an identity of the second fingerprint (e.g., with the determining unit).

4600 4600 4616 4608 4600 4620 4600 4608 4616 In some embodiments, the first fingerprint and the second fingerprint are detected while a first application is an active application at the first device; and after sending a response to the second electronic device that enables the second electronic device to be configured as an auxiliary display for the first electronic device(e.g., with the sending unit), the processing unitis further configured to configure the second electronic device as an auxiliary display for the first device(e.g., with the configuring unit), and while the second electronic device is configured as an auxiliary display for the first device, the processing unitis further configured to send instructions to the second device (e.g., with the sending unit) that enable the second device to display a set of one or more user interface elements associated with the first application.

In some embodiments, where the set of one or more user interface elements associated with the first application include one or more controls for performing operations in the first application.

4602 4600 In some embodiments, where the set of one or more user interface elements associated with the first application include a portion of a workspace that is a continuation of a workspace displayed on the display unitof the first device.

4600 4608 4612 4602 4608 4622 In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the processing unitis further configured to receive input data indicative of inputs detected by the second device (e.g., with the receiving unit), where the inputs correspond to interactions with one or more of the user interface elements associated with the first application that are displayed on the display unitof the second device, and the processing unitis further configured to perform operations associated with the first application in accordance with the input data (e.g., with the performing unit).

4600 4608 4612 4608 4622 In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the processing unitis further configured to receive operation data from the second electronic device that specifies one or more operations corresponding to interactions with one or more of the user interface elements associated with the first application that are displayed on the display unit of the second device (e.g., with the receiving unit), and the processing unitis further configured to perform the one or more operations (e.g., with the performing unit).

4600 4608 4604 4600 4610 4608 4612 4608 4602 4600 4624 In some embodiments, while the second electronic device is configured as the auxiliary display for the first device, the processing unitis further configured to detect a first gesture on a touch-sensitive surface unitof the first device(e.g., with the detecting unit). The processing unitis further configured receive a communication from the second device indicative of a second gesture detected on a touch-sensitive surface unit of the second device (e.g., with the receiving unit), and in response to detecting the first gesture and receiving the communication indicative of the second gesture, the processing unitis further configured to move a user interface object associated with the first application between the display unitof the first deviceand the display unit of the second device (e.g., with the moving unit).

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

45 45 FIGS.A-D 1 1 FIGS.A-B 46 FIG. 1 1 FIGS.A-B 4502 4538 4504 4530 4534 4540 4508 4516 4518 4514 4524 4544 4522 4532 4536 4542 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, detecting operationsand, receiving operations,,, and, determining operations,, and, sending operationsand, forgoing operation, configuring operation, performing operationsand, and moving operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

Many electronic devices enable the configuration of audio elements based on proximity sensor data. For example, based on information from a proximity sensor a mobile telephone device can switch between a receiver mode and a speaker mode. The device described below improves on existing methods by utilizing a detected change in orientation of a fingerprint to select a set of beamforming parameters from a plurality of beamforming parameters so as to more efficiently operate a set of one or more audio elements.

While a first fingerprint is detected in a first orientation on a fingerprint sensor, the device operates a set of one or more audio elements in accordance with a first set of beamforming parameters. Then, the device detects a subsequent fingerprint having a second orientation different from the first orientation on the fingerprint sensor. The subsequent fingerprint is selected from a set consisting of the first fingerprint with a changed orientation and a second fingerprint distinct from the first fingerprint. Finally, in response to detecting the subsequent fingerprint having the second orientation on the fingerprint sensor, the device operates the set of one or more audio elements in accordance with a second set of beamforming parameters different from the first set of beamforming parameters.

450 451 100 112 167 169 359 1 451 112 359 2 451 112 169 359 1 359 2 169 112 169 359 1 359 2 112 450 451 359 1 359 2 359 1 359 2 450 1 FIG.A 47 47 48 48 FIGS.A-I, andA-B 4 FIG.B 4 FIG.B 47 47 48 48 FIGS.A-L andA-B 47 47 FIGS.A-L 47 47 FIGS.A-L In some embodiments, the device is an electronic device with a separate display (e.g., display) and a separate touch-sensitive surface (e.g., touch-sensitive surface). In some embodiments, the device is portable multifunction device, the display is touch screen, and the touch-sensitive surface includes tactile output generatorson the display (). The device described below with reference toincludes one or more fingerprint sensors. In some embodiments, the one or more fingerprint sensors include one or more integrated fingerprint sensors-() that are integrated in to the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). In some embodiments, the one or more fingerprint sensors include separate fingerprint sensors-() that are separate from the touch-sensitive surface (e.g., separate touch-sensitive surfaceor touch sensitive display system). Unless specified otherwise, a fingerprint sensordescribed below is, optionally, either an integrated fingerprint sensor-or a separate fingerprint sensor-, depending on the configuration of the hardware and software of the device that includes the fingerprint sensor. For convenience of explanation, the embodiments described with reference towill be discussed with reference to touch screenand fingerprint sensor(e.g., an integrated fingerprint sensor-or a separate fingerprint sensor-); in such embodiments, the focus selector is, optionally: a respective contact, a representative point corresponding to a contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on the touch screen, in place of a cursor. Analogous operations are, optionally, performed on a device with display, a separate touch-sensitive surface, and an integrated fingerprint sensor-or a separate fingerprint sensor-in response to detecting the inputs described inon integrated fingerprint sensor-or separate fingerprint sensor-, while displaying the user interfaces shown inon display.

47 47 FIGS.A-C 47 47 FIGS.A-C 47 47 FIGS.A-C 47 47 FIGS.A-C 47 47 FIGS.A-C 47 47 FIGS.A-C 100 100 111 4701 112 4704 4706 169 illustrate a portable multifunction deviceoperating in an audio generation mode. While in audio generation mode, portable multifunction devicegenerates audio via speaker. A media player (e.g., Apple, Inc.'s iTunes application) is displayed on user interfaceof touch screenin. The media player illustrated inis, for example, playing an audio file, or streaming audio.further illustrate fingerprint axiswhich indicates the angle of a respective fingerprint and fingerprint sensor axiswhich indicates the angle of fingerprint sensor.are alike in all respects save the orientation of the respective fingerprint; thus, only the differences betweenwill be discussed for the sake of brevity.

47 FIG.A 47 FIG.A 47 FIG.A 4702 169 204 100 359 1 4702 4702 4704 169 4706 4702 169 illustrates a first fingerprintin a first orientation over fingerprint sensor, which is integrated into home buttonof device(e.g., integrated fingerprint sensor-). Fingerprintis, for example, a user's right thumb.illustrates the angle of fingerprintvia fingerprint axisat 90 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 0 degrees.

47 FIG.B 47 FIG.B 47 FIG.B 4702 169 4702 4704 169 4706 4702 169 illustrates first fingerprintin a second orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 135 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 45 degrees.

47 FIG.C 47 FIG.C 47 FIG.C 4702 169 4702 4704 169 4706 4702 169 illustrates first fingerprintin a third orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 180 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 90 degrees.

47 47 FIGS.D-F 47 47 FIGS.D-F 47 47 FIGS.D-F 47 47 FIGS.D-F 100 100 113 illustrate a portable multifunction deviceoperating in an audio capture mode. While in audio capture mode, portable multifunction devicecaptures audio via microphone.illustrate, for example, Apple, Inc.'s Voice Memos application which records (or captures) audio signals for subsequent playback or transmission.are alike in all respects save the orientation of the respective fingerprint; thus, only the differences betweenwill be discussed for the sake of brevity.

47 FIG.D 47 FIG.D 47 FIG.D 4708 169 204 100 359 1 4708 4708 4704 169 4706 4708 169 illustrates a first fingerprintin a first orientation over fingerprint sensor, which is integrated into home buttonof device(e.g., integrated fingerprint sensor-). Fingerprintis, for example, a user's left thumb.illustrates the angle of fingerprintvia fingerprint axisat 90 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 0 degrees.

47 FIG.E 47 FIG.E 47 FIG.E 4708 169 4708 4704 169 4706 4708 169 illustrates first fingerprintin a second orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 45 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 45 degrees.

47 FIG.F 47 FIG.F 47 FIG.F 4708 169 4708 4704 169 4706 4708 169 illustrates first fingerprintin a third orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 0 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 90 degrees.

47 47 FIGS.G-I 47 47 FIGS.G-I 47 47 FIGS.G-I 47 47 FIGS.G-I 100 4701 112 4712 4712 100 113 111 illustrate a portable multifunction deviceoperating in a speaker phone mode. A plurality of buttons are displayed on user interfaceof touch screenincluding a mute button, a display keypad button, an activate speaker button, an add call button, an activate FaceTime button, a display contacts button and an end call button. In, speaker buttoncorresponding to speaker phone mode is activated. While in speaker phone mode, portable multifunction devicecaptures audio via microphoneand generates audio via speaker.are alike in all respects save the orientation of the respective fingerprint; thus, only the differences betweenwill be discussed for the sake of brevity.

47 FIG.G 47 FIG.G 47 FIG.G 4710 169 204 100 4710 4710 4704 169 4706 4710 169 illustrates a first fingerprintin a first orientation over fingerprint sensor, which is integrated into home buttonof device. Fingerprintis, for example, a user's right thumb.illustrates the angle of fingerprintvia fingerprint axisat 90 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 0 degrees.

47 FIG.H 47 FIG.H 47 FIG.H 4710 169 4710 4704 169 4706 4710 169 illustrates first fingerprintin a second orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 135 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 45 degrees.

47 FIG.I 47 FIG.I 47 FIG.I 4710 169 4710 4704 169 4706 4710 169 illustrates first fingerprintin a third orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 180 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 90 degrees.

47 47 FIGS.J-L 47 47 FIGS.J-L 47 47 FIGS.J-L 47 47 FIGS.J-L 100 112 100 112 100 113 111 illustrate a portable multifunction deviceoperating in a video conference or video chat mode (e.g., Apple, Inc.'s FaceTime application).illustrate displaying a video chat participant on touch screen. Furthermore, a forward facing camera (not shown) enabled to capture video relative to the side of deviceassociated with touch screenis configured to capture video of a user so as to transmit the video to the chat participant. While in video chat mode, portable multifunction devicecaptures audio via microphone, captures video via the forward facing camera (not shown) and generates audio via speaker.are alike in all respects save the orientation of the respective fingerprint(s); thus, only the differences betweenwill be discussed for the sake of brevity.

47 FIG.J 47 FIG.J 47 FIG.J 4714 169 204 100 4714 4714 4704 169 4706 4714 169 illustrates a first fingerprintin a first orientation over fingerprint sensor, which is integrated into home buttonof device. Fingerprintis, for example, a user's right thumb.illustrates the angle of fingerprintvia fingerprint axisat 90 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of first fingerprintrelative to the axis of fingerprint sensoris 0 degrees.

47 FIG.K 47 FIG.K 47 FIG.K 4716 169 4716 4716 4704 169 4706 4716 169 illustrates a second fingerprintin a second orientation over fingerprint sensor. Fingerprintis, for example, a user's right index finger.illustrates the angle of fingerprintvia fingerprint axisat 135 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of second fingerprintrelative to the axis of fingerprint sensoris 45 degrees.

47 FIG.L 47 FIG.L 47 FIG.L 4716 169 4716 4704 169 4710 4716 169 illustrates second fingerprintin a third orientation over fingerprint sensor.illustrates the angle of fingerprintvia fingerprint axisat 180 degrees and the angle of fingerprint sensorvia fingerprint sensor axisat 90 degrees. Thus, in, the angle of second fingerprintrelative to the axis of fingerprint sensoris 90 degrees.

48 48 FIGS.A-B 3 FIG. 1 FIG.A 4800 4800 300 100 4800 are flow diagrams illustrating a methodof changing beamforming parameters based on fingerprint orientation in accordance with some embodiments. The methodis performed at an electronic device (e.g., device,, or portable multifunction device,) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in methodare, optionally, combined and/or the order of some operations is, optionally, changed.

4800 As described below, the methodprovides an intuitive way to change beamforming parameters based on fingerprint orientation. The method reduces the cognitive burden on a user when changing beamforming parameters, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to change beamforming parameters based on fingerprint orientation faster and more efficiently conserves power and increases the time between battery charges.

4802 4702 4706 169 100 111 100 4702 4706 47 FIG.A 47 FIG.A While a first fingerprint is detected in a first orientation on a fingerprint sensor, the device operates () a set of one or more audio elements in accordance with a first set of beamforming parameters., for example, shows a first fingerprintin a first orientation (e.g., a 0 degree relative angle to fingerprint sensor axis) on fingerprint sensor., for example, shows deviceoperating speaker(e.g., one of a set of one or more audio elements of device) in accordance with a first set of beamforming parameters while first fingerprintis at a 0 degree angle relative to fingerprint sensor axis.

4804 100 111 112 47 47 FIGS.A-C In some embodiments, the set of one or more audio elements includes () a plurality of speakers. For example, the change in beamforming parameters changes the times at which audio is generated by the speakers so as to produce constructive interference that amplifies the sound of the speakers at a predefined location, along a predefined path, or within a predefined region relative to the speakers., for example, show deviceoperating speaker(e.g., one of a plurality of speakers) to generate sound corresponding to an audio file, or streaming audio, selected by a user via a media player (e.g., Apple, Inc.'s iTunes application) displayed on touch screen.

4806 100 113 47 47 FIGS.D-F In some embodiments, the set of one or more audio elements includes () a plurality of microphones. For example, the change in beamforming parameters changes the times at which sound received by the microphones is processed so as to produce constructive interference that amplifies the sound received by the microphones coming from a predefined location, along a predefined path, or within a predefined region relative to the microphones., for example, show deviceoperating microphone(e.g., one of a plurality of microphones) to capture sound while in an audio recording mode (e.g., via Apple, Inc.'s Voice Memos application).

4808 100 111 113 100 111 113 47 47 FIGS.G-I 47 47 FIGS.J-L In some embodiments, the set of one or more audio elements includes () one or more microphones and one or more speakers., for example, show deviceoperating speaker(e.g., one of a plurality of speakers) and microphone(e.g., one of a plurality of microphones) to generate and capture sound while in a speaker phone mode., for example, show deviceoperating speaker(e.g., one of a plurality of speakers) to generate sound and microphone(e.g., one of a plurality of microphones) to capture sound while in a video chat mode (e.g., via Apple, Inc.'s FaceTime application).

4810 The device detects () a subsequent fingerprint having a second orientation different from the first orientation on the fingerprint sensor, where the subsequent fingerprint is selected from the set consisting of the first fingerprint with a changed orientation and a second fingerprint distinct from the first fingerprint.

47 FIG.B 47 FIG.A 47 FIG.B 47 FIG.A 47 FIG.A 47 FIG.B 47 FIG.A 47 FIG.B 100 4702 4706 169 4706 4702 4702 4706 4702 4702 169 , for example, shows devicedetecting a subsequent fingerprinthaving a second orientation (e.g., a 45 degree angle relative to fingerprint sensor axis) on fingerprint sensordifferent from the first orientation in(e.g., a 0 degree angle relative to fingerprint sensor axis). Subsequent fingerprintinis first fingerprintinwith a changed orientation (e.g., a 45 degree angle as opposed to a 0 degree angle relative to fingerprint sensor axis). In this example, subsequent fingerprintis the user's right thumb. According to some embodiments, subsequent fingerprint, however, corresponds to one of either two scenarios: (1) continuous contact of the user's right thumb with fingerprint sensorbetween the first orientation inand the second orientation in, or (2) after detection of the user's right thumb in the first orientation in, lift off of the user's right thumb and placement of the user's right thumb in the second orientation in.

47 FIG.K 47 FIG.J 47 FIG.K 47 FIG.J 47 FIG.J 47 FIG.K 47 FIG.J 47 FIG.K 100 4716 4706 169 4706 4716 4714 4714 169 4716 169 169 169 , for example, shows devicedetecting a subsequent fingerprinthaving a second orientation (e.g., a 45 degree angle relative to fingerprint sensor axis) on fingerprint sensordifferent from the first orientation in(e.g., a 0 degree angle relative to fingerprint sensor axis). Subsequent fingerprintinis a second fingerprint distinct from first fingerprintin. In this example, fingerprintdetected on fingerprint sensorinis a user's right thumb, and fingerprintdetected on fingerprint sensorinis a user's right index finger. Thus, in this example, after detection of the user's right thumb in, the user's right thumb was lifted off of fingerprint sensor, and, subsequently, the user's right index finger is detected on fingerprint sensorin.

4812 4702 4706 169 100 111 100 4702 4706 47 FIG.B 47 FIG.B In response to detecting the subsequent fingerprint having the second orientation on the fingerprint sensor, the device operates () the set of one or more audio elements in accordance with a second set of beamforming parameters different from the first set of beamforming parameters. In some embodiments, the change in the beamforming parameters changes constructive and destructive interference of audio received/generated by a plurality of audio elements (e.g., speakers or microphones) so as to extend the effective range of the audio elements., for example, shows a subsequent fingerprintin a second orientation (e.g., a 45 degree relative to fingerprint sensor axis) on fingerprint sensor., for example, shows deviceoperating speaker(e.g., one of a set of one or more audio elements of device) in accordance with a second set of beamforming parameters while subsequent fingerprintis at the 45 degree relative to fingerprint sensor axis.

4814 In some embodiments, in response to detecting the change in orientation of the fingerprint on the fingerprint sensor, the device selects () the second set of beamforming parameters from a plurality of sets of predefined beamforming parameters associated with corresponding fingerprint orientations. For example, a first set of beamforming parameters is optimized for situations where the fingerprint is at 0 degrees on the fingerprint sensor (e.g., an axis of the fingerprint is aligned with a predefined axis of the fingerprint sensor), a second set of beamforming parameters is optimized for situations where the angle of the fingerprint is 90 degrees on the fingerprint sensor relative to the predefined axis of the fingerprint sensor, etc. In some embodiments, the beamforming parameters are stored at the device, and the beamforming parameters are retrieved from storage in response to detecting the change in orientation. In some embodiments, the beamforming parameters are generated by the device as needed. In some embodiments, the plurality of sets of predefined beamforming parameters correspond to fingerprint orientations at regularly spaced intervals, such as 15 degrees, 30 degrees or 45 degrees. For example, when an angle of the fingerprint is 45 degrees on the fingerprint sensor relative to the predefined axis of the fingerprint sensor, the device combines contributions from different sets of baseline beamforming parameters (e.g., a set of 30 degree beamforming parameters and a set of 60 degree beamforming parameters) in accordance with an angle of the fingerprint on the fingerprint sensor.

4816 4818 100 100 113 169 47 47 FIGS.D-F In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate () for a change in the likely location of the mouth of a user () of the device relative to the audio elements of the device (e.g., the change in beamforming captures sound from the user more effectively)., for example, show devicecompensating for a change in the likely location of the mouth of a user of devicerelative to microphonewhile capturing sound in a sound recording mode by changing the beamforming parameters according to a change in orientation of the fingerprint on fingerprint sensor.

4816 4820 100 100 111 169 47 47 FIGS.A-C In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate () for a change in the likely location of an car of a user () of the device relative to the audio elements of the device (e.g., the change in beamforming delivers sound to the user more effectively)., for example, show devicecompensating for a change in the likely location of the car of a user of devicerelative to speakerwhile generating sound in media playback mode by changing the beamforming parameters according to a change in orientation of the fingerprint on fingerprint sensor.

4816 4822 100 100 111 113 169 47 47 FIGS.G-I In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate () for a change in the likely location of a hand of a user () of the device relative to the audio elements of the device (e.g., the change in beamforming captures sound from the user and/or delivers sound to the user more effectively if a hand is likely to be blocking at least a part of a path between the audio elements and the user's cars/mouth)., for example, show devicecompensating for a change in the likely location of the hand of a user of devicerelative to speakerand microphonewhile generating and capturing sound in speaker phone mode by changing the beamforming parameters according to a change in orientation of the fingerprint on fingerprint sensor.

48 48 FIGS.A-B 48 48 FIGS.A-B 4800 4800 It should be understood that the particular order in which the operations inhave been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., those methods described above) are also applicable in an analogous manner to methoddescribed above with respect to. For example, the fingerprints described above with reference to methodoptionally have one or more of the characteristics of the fingerprints described herein with reference to other methods described herein (e.g., those methods described above). For brevity, these details are not repeated here.

49 FIG. 49 FIG. 4900 In accordance with some embodiments,shows a functional block diagram of an electronic deviceconfigured in accordance with the principles of the various described embodiments. The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described inare, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

49 FIG. 4900 4902 4904 4906 4908 4902 4904 4906 4908 4910 4912 4914 As shown in, an electronic deviceincludes a display unitconfigured to display one or more user interface objects; a fingerprint sensor unit; a set of one or more audio unitsconfigured to generate and/or capture audio signals; and a processing unitcoupled to the display unit, the fingerprint sensor unit, and the set of one or more audio units. In some embodiments, the processing unitincludes an operating unit, a detecting unitand a selecting unit.

4912 4904 4908 4910 4906 4908 4912 4904 4904 4910 4906 While a first fingerprint is detected (e.g., with the detecting unit) in a first orientation on the fingerprint sensor unit, the processing unitis configured to operate (e.g., with the operating unit) the set of one or more audio unitsin accordance with a first set of beamforming parameters. The processing unitis further configured to: detect (e.g., with detecting unit) a subsequent fingerprint having a second orientation different from the first orientation on the fingerprint sensor unit, where the subsequent fingerprint is selected from a set consisting of the first fingerprint with a changed orientation and a second fingerprint distinct from the first fingerprint; and in response to detecting the subsequent fingerprint having the second orientation on the fingerprint sensor unit, operate (e.g., with the operating unit) the set of one or more audio unitsin accordance with a second set of beamforming parameters different from the first set of beamforming parameters.

4906 4906 4906 In some embodiments, the set of one or more audio unitsincludes a plurality of speakers. In some embodiments, the set of one or more audio unitsincludes a plurality of microphones. In some embodiments, the set of one or more audio unitsincludes one or more microphones and one or more speakers.

4908 4914 4912 4904 In some embodiments, the processing unitis further configured to select (e.g., with the selecting unit) the second set of beamforming parameters from a plurality of sets of predefined beamforming parameters associated with corresponding fingerprint orientations to in response to detecting (e.g., with the detecting unit) the subsequent fingerprint having the second orientation on the fingerprint sensor unit.

4906 4906 4906 In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate for a change in the likely location of the mouth of a user of the device relative to the one or more audio units. In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate for a change in the likely location of an ear of a user of the device relative to the one or more audio units. In some embodiments, differences between the first set of beamforming parameters and the second set of beamforming parameters compensate for a change in the likely location of a hand of a user of the device relative to the one or more audio units.

1 3 FIGS.A and The operations in the information processing methods described above are, optionally implemented by running one or more functional modules in information processing apparatus such as general purpose processors (e.g., as described above with respect to) or application specific chips.

48 48 FIGS.A-B 1 1 FIGS.A-B 49 FIG. 1 1 FIGS.A-B 4802 4812 4810 4814 170 180 190 171 170 112 174 136 1 180 136 1 186 180 190 190 176 177 192 190 178 The operations described above with reference toare, optionally, implemented by components depicted inor. For example, operating operationsand, detecting operation, and selecting operationare, optionally, implemented by event sorter, event recognizer, and event handler. Event monitorin event sorterdetects a contact on touch-sensitive display, and event dispatcher moduledelivers the event information to application-. A respective event recognizerof application-compares the event information to respective event definitions, and determines whether a first contact at a first location on the touch-sensitive surface corresponds to a predefined event or sub-event, such as selection of an object on a user interface. When a respective predefined event or sub-event is detected, event recognizeractivates an event handlerassociated with the detection of the event or sub-event. Event handleroptionally utilizes or calls data updateror object updaterto update the application internal state. In some embodiments, event handleraccesses a respective GUI updaterto update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best use the invention and various described embodiments with various modifications as are suited to the particular use contemplated.