Device pairing

This is a reissue application of U.S. Pat. No. 11,259,064, which was filed as U.S. patent application Ser. No. 16/676,266 on Nov. 6, 2019, and issued on Feb. 22, 2022, and which is a continuation application of U.S. patent application Ser. No. 15/867,258, filed Jan. 10, 2018, now U.S. Pat. No. 10,516,910, issued on Dec. 24, 2019, which is a continuation application of U.S. patent application Ser. No. 15/171,100, filed Jun. 2, 2016, now U.S. Pat. No. 9,948,974, issued on Apr. 17, 2018, which is a continuation application of U.S. patent application Ser. No. 14/102,612, filed on Dec. 11, 2013, now U.S. Pat. No. 9,361,541, issued Jun. 7, 2016, the disclosures of which are incorporated herein by reference in their entirety.

This application is a continuation application of U.S. patent application Ser. No. 15/867,258, filed Jan. 10, 2018, which is a continuation application of U.S. patent application Ser. No. 15/171,100, filed Jun. 2, 2016, that is now U.S. Pat. No. 9,948,974, which is a continuation application of U.S. patent application Ser. No. 14/102,612, filed Dec. 11, 2013, that is now U.S. Pat. No. 9,361,541, which are each incorporated herein by reference in their entirety.

One or more embodiments relate generally to device pairing and, in particular, to automated pairing between multiple devices.

For pairing a hand-held (HH) device to a television device (TV) within a home (assuming that both of devices are using the same access point (AP)), typically the HH device presents a user with a list of all TV devices on the home network. The user would then have to select one of the devices in the list. The TV devices, however, do not know which user is currently in front of a particular TV device.

Some embodiments provide a method for wirelessly connecting a first device with an external second device includes wirelessly communicating between the first device and the external second device. Based on the wireless communication, the first device causes displaying of a dynamic graphic pattern on a display of the external second device. The dynamic graphic pattern having a shape that changes over time. A camera of the first device receives the dynamic graphic pattern. The first device recognizes the dynamic graphic pattern associated with the external second device. Data is wirelessly transferred between the first device and the external second device based on the recognized dynamic graphic pattern.

Some embodiments provide a first electronic device that includes a processor and a memory storing one or more programs for execution of the processor. The one or more programs including instructions to wirelessly communicate with an external second device; based on the wireless communication, cause a dynamic graphic pattern to be displayed on a display of the external second device, the dynamic graphic pattern having a shape that changes over time; receive, by a camera of the first electronic device, the dynamic graphic pattern; recognize the dynamic graphic pattern associated with the external second device; and wirelessly transfer data between the first electronic device and the external second device based on the recognized dynamic graphic pattern.

Some embodiments provide a non-transitory processor-readable medium that includes a program that when executed by a processor performs a method comprising: wirelessly communicating between the first device and the external second device. Based on the wireless communication, the first device causing a dynamic graphic pattern to be displayed on a display of the external second device. The dynamic graphic pattern having a shape that changes over time. A camera of the first device receives the dynamic graphic pattern. The first device recognizes the dynamic graphic pattern associated with the external second device. Data is wirelessly transferred between the first device and the external second device based on the recognized dynamic graphic pattern.

These and other aspects and advantages of the embodiments will become apparent from the following detailed description, which, when taken in conjunction with the drawings, illustrate by way of example the principles of the embodiments.

The following description is made for the purpose of illustrating the general principles of the embodiments and is not meant to limit the inventive concepts claimed herein. Further, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations. Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including meanings implied from the specification as well as meanings understood by those skilled in the art and/or as defined in dictionaries, treatises, etc.

One or more embodiments of relate generally to device and facial pairing. In one embodiment, a method for wirelessly connecting a first device with an external second device includes wirelessly communicating between the first device and the external second device. Based on the wireless communication, the first device causes displaying of a dynamic graphic pattern on a display of the external second device. The dynamic graphic pattern having a shape that changes over time. A camera of the first device receives the dynamic graphic pattern. The first device recognizes the dynamic graphic pattern associated with the external second device. Data is wirelessly transferred between the first device and the external second device based on the recognized dynamic graphic pattern.

In one embodiment, the electronic devices comprise one or more mobile electronic devices capable of data communication over a communication link such as a wireless communication link. Examples of such mobile device include a mobile phone or smart phone device, a mobile tablet device, a wearable device, a mobile computing device, etc.

1 FIG. 1 FIG. 10 12 110 12 110 10 12 11 10 12 11 is a schematic view of a communications system in accordance with one embodiment. Communications systemmay include a communications device that initiates an outgoing communications operation (transmitting/initiator device) and communications network, which transmitting devicemay use to initiate and conduct communications operations with other communications devices within communications network. For example, communications systemmay include a communication device that receives the communications operation from the transmitting device(receiving device/listener). Although communications systemmay include several transmitting devicesand receiving devices, only one of each is shown into simplify the drawing.

110 110 110 110 12 11 110 13 12 11 Any suitable circuitry, device, system or combination of these (e.g., a wireless communications infrastructure including communications towers and telecommunications servers) operative to create a communications network may be used to create communications network. Communications networkmay be capable of providing communications using any suitable communications protocol. In some embodiments, communications networkmay support, for example, traditional telephone lines, cable television, Wi-Fi (e.g., a 802.11 protocol), BLUETOOTH®, high frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz communication systems), infrared, other relatively localized wireless communication protocol, or any combination thereof. In some embodiments, communications networkmay support protocols used by wireless and cellular phones and personal email devices (e.g., a) BLACKBERRY®). Such protocols can include, for example, GSM, GSM plus EDGE, CDMA, quadband, and other cellular protocols. In another example, a long-range communications protocol can include Wi-Fi and protocols for placing or receiving calls using VOIP or LAN. Transmitting deviceand receiving device, when located within communications network, may communicate over a bidirectional communication path such as path. Both transmitting deviceand receiving devicemay be capable of initiating a communications operation and receiving an initiated communications operation.

12 11 12 11 Transmitting deviceand receiving devicemay include any suitable device for sending and receiving communications operations. For example, transmitting deviceand receiving devicemay include a media player, a cellular telephone or a landline telephone, a personal e-mail or messaging device with audio and/or video capabilities, pocket-sized personal computers such as an iPAQ Pocket PC available by Hewlett Packard Inc., of Palo Alto, Calif., personal digital assistants (PDAs), a desktop computer, a laptop computer, and any other device capable of communicating wirelessly (with or without the aid of a wireless enabling accessory system) or via wired pathways (e.g., using traditional telephone wires). The communications operations may include any suitable form of communications, including for example, voice communications (e.g., telephone calls), data communications (e.g., e-mails, text messages, media messages), or combinations of these (e.g., video conferences).

2 FIG. 100 1 140 100 120 130 110 120 121 122 120 120 125 1126 2127 128 shows a functional block diagram of an embodiment of an architecture systemfor application connection by hand held (HH) electronic devices-N(N being a positive integer), according to an embodiment. In one embodiment, the systemcomprises a television device(e.g., a TV, a computer, tablet, etc.), a cloud or server deviceand network connection device. In one embodiment, the television devicecomprises a display, a camera or imaging device(either integrated with the TVor separate from the TV), and an application manager modulefor managing applications, such as application, applicationup to application N, N being a positive integer.

121 120 120 122 120 120 110 130 120 In one embodiment, the displaymay be a separate device from the television deviceor integrated with the television device. In one embodiment, the cameramay be a separate device from the television deviceor integrated with the television device. In one embodiment, the network connection devicemay comprise a network interface, such as a network modem, router, etc. for handling communications between the television device and the server deviceand for forming a local network that the television devicemay be connected with.

12 11 1 140 1 140 141 142 143 144 145 146 In one embodiment, both transmitting deviceand receiving devicemay include some or all of the features of electronics devices-N. In one embodiment, the electronic devices-Nmay comprise a display, input mechanism, communications circuitry, control circuitry, a camera, and a global positioning system (GPS) receiver module, a microphone, audio output, and any other suitable components.

141 142 143 144 1 140 In one embodiment, all of the applications employed by display, input mechanism, the audio output and communications circuitrymay be interconnected and managed by control circuitry. In one example, a hand-held music player capable of transmitting music to other tuning devices may be incorporated into the electronics devices-N.

1 140 1 140 1 140 1 150 In one embodiment, the audio output may include any suitable audio component for providing audio to a user of any of the electronics devices-N. For example, the audio output may include one or more speakers (e.g., mono or stereo speakers) built into an electronics device-N. In some embodiments, the audio output may include an audio component that is remotely coupled to an electronics device-N. For example, the audio output may include a headset, headphones or earbuds that may be coupled to communications device with a wire (e.g., coupled to an electronics device-Nwith a jack) or wirelessly (e.g., BLUETOOTH® headphones or a BLUETOOTH® headset).

141 141 1 140 141 1 140 141 144 In one embodiment, displaymay include any suitable screen or projection system for providing a display visible to the user. For example, displaymay include a screen (e.g., an LCD screen) that is incorporated in an electronics device-N. As another example, displaymay include a movable display or a projecting system for providing a display of content on a surface remote from an electronics device-N(e.g., a video projector). Displaymay be operative to display content (e.g., information regarding communications operations or information regarding available media selections) under the direction of control circuitry.

142 1 140 142 142 In one embodiment, input mechanismmay be any suitable mechanism or user interface for providing user inputs or instructions to an electronics device-N. Input mechanismmay take a variety of forms, such as a button, keypad, dial, a click wheel, or a touch screen. The input mechanismmay include a multi-touch screen. The input mechanism may include a user interface that may emulate a rotary phone or a multi-button keypad, which may be implemented on a touch screen or the combination of a click wheel or other user input device and a screen.

143 110 1 140 143 1 FIG. In one embodiment, communications circuitrymay be any suitable communications circuitry operative to connect to a communications network (e.g., communications network,) and to transmit communications operations and media from an electronics device-Nto other devices within the communications network. Communications circuitrymay be operative to interface with the communications network using any suitable communications protocol such as, for example, Wi-Fi (e.g., an 802.11 protocol), BLUETOOTH®, high frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz communication systems), infrared, GSM, GSM plus EDGE, CDMA, quadband, and other cellular protocols, VOIP, or any other suitable protocol.

143 143 143 1 140 In some embodiments, communications circuitrymay be operative to create a communications network using any suitable communications protocol. For example, communications circuitrymay create a short-range communications network using a short-range communications protocol to connect to other communications devices. For example, communications circuitrymay be operative to create a local communications network using the BLUETOOTH® protocol to couple an electronics device-Nwith a BLUETOOTH® headset.

144 1 140 144 1 140 1 140 1 140 In one embodiment, control circuitrymay be operative to control the operations and performance of an electronics device-N. Control circuitrymay include, for example, a processor, a bus (e.g., for sending instructions to the other components of an electronics device-N), memory, storage, or any other suitable component for controlling the operations of an electronics device-N. In some embodiments, a processor may drive the display and process inputs received from the user interface. The memory and storage may include, for example, cache, Flash memory, ROM, and/or RAM. In some embodiments, memory may be specifically dedicated to storing firmware (e.g., for device applications such as an operating system, user interface functions, and processor functions). In some embodiments, memory may be operative to store information related to other devices with which an electronics device-Nperforms communications operations (e.g., saving contact information related to communications operations or storing information related to different media types and media items selected by the user).

144 1 140 1 140 1 140 1 140 In one embodiment, the control circuitrymay be operative to perform the operations of one or more applications implemented on an electronics device-N. Any suitable number or type of applications may be implemented. Although the following discussion will enumerate different applications, it will be understood that some or all of the applications may be combined into one or more applications. For example, an electronics device-Nmay include an application connection application, a dialog application, a map application, a media application (e.g., QuickTime, MobileMusic.app, or MobileVideo.app). In some embodiments, an electronics device-Nmay include one or several applications operative to perform communications operations. For example, an electronics device-Nmay include a messaging application, a mail application, a telephone application, a voicemail application, an instant messaging application (e.g., for chatting), a videoconferencing application, a fax application, or any other suitable application for performing any suitable communications operation.

1 140 1 140 1 140 1 140 In some embodiments, an electronics device-Nmay include one or more microphones. For example, an electronics device-Nmay include a microphone to allow the user to transmit audio (e.g., voice audio) during a communications operation or as a means of establishing a communications operation or as an alternate to using a physical user interface. A microphone may be incorporated in an electronics device-N, or may be remotely coupled to an electronics device-N. For example, a microphone may be incorporated in wired headphones, or a microphone may be incorporated in a wireless headset.

1 140 1 140 In one embodiment, an electronics device-Nmay include any other component suitable for performing a communications operation. For example, an electronics device-Nmay include a power supply, ports or interfaces for coupling to a host device, a secondary input mechanism (e.g., an ON/OFF switch), or any other suitable component.

1 140 In one embodiment, a user may direct an electronics device-Nto perform a communications operation using any suitable approach. As one example, a user may receive a communications request from another device (e.g., an incoming telephone call, an email or text message, an instant message), and may initiate a communications operation by accepting the communications request. As another example, the user may initiate a communications operation by identifying another communications device and transmitting a request to initiate a communications operation (e.g., dialing a telephone number, sending an email, typing a text message, or selecting a chat screen name and sending a chat request).

1 140 146 145 146 In one embodiment, an electronic device-N(N being a positive integer) may comprise a mobile device that may utilize mobile device hardware functionality including: the GPS receiver module, the camera, a compass module, and an accelerometer and gyroscope module. The GPS receiver modulemay be used to identify a current location of the mobile device (i.e., user). The compass module is used to identify direction of the mobile device. The accelerometer and gyroscope module is used to identify tilt of the mobile device.

100 1 140 120 1126 2127 128 1126 2127 128 120 121 The systemprovides the electronic devices-N(N being a positive integer) the ability to connect to an application launched on the television device, such as application, applicationto application N. In one embodiment, the application, applicationto application Nmay comprise software applications executing on the television device, or executing on another device and having a visual display portion of the application shown on the display.

120 1 140 1 140 1 140 1 140 In one embodiment, the local network (e.g., a wireless network) that the television deviceis connected to is a secured network with a security code (e.g., pass code, password, key code, etc.) providing access to the local network such that without having knowledge of the key code, the electronic devices-Ncannot pair or join the local network. In one embodiment, access is provided to the electronic devices-Nto the local network that the application is launched by means for connecting, such as by obtaining information from a server based on location of an electronic device (e.g., electronic devices-N), obtaining information from a communication (e.g., text message, email, chat, etc.), etc. In one embodiment, once an electronic device-Nis connected to the local network, the connection may have a limited life for security purposes (e.g., one hour, two hours, 4 hours, etc.). In one embodiment, the security code may be unique for each session.

3 FIG. 300 140 120 120 320 140 shows an example scenariofor device pairing, according to an embodiment. One or more embodiments provide for a user at home to pair their HH electronic devicewith a TVwhere there is a possibility of multiple TVsin the household or other environment. In one or more embodiments, in the case the HH device is used by multiple family/community members, the pairing process associates a particular user (e.g., user) with the HH electronic device.

300 120 140 300 140 120 140 120 300 122 122 141 140 300 122 120 In one embodiment, for scenarioit is assumed that multiple TVsand multiple HH electronic devicesare present in the environment (e.g., a home environment). In one embodiment, for scenario, the HH electronic deviceand TVare on the same network (or access point (AP)), which may be a particular home setting and the HH electronic deviceand TVhave been paired. In one embodiment, it may be assumed for scenariothat the TV camerais constantly working (or may be started by a command, such as a voice command, remote control (RC) command, etc.), and the TV camerais able to recognize a characteristic (e.g., color, pattern, shape, etc.) of an image or object (e.g., a blob, design, photograph, pattern, etc.) on the displayof the HH electronic device. In one embodiment, for scenarioit is assumed that the TV cameraand other processing by the TVis able to recognize and keep track of a user face (e.g., via facial recognition techniques).

300 140 140 120 120 311 310 141 140 122 141 122 310 311 In one embodiment, scenarioidentifies which HH electronic device(of multiple HH electronic devices) is desired to be paired with which TV(of multiple TVs). In one embodiment, a guest or user launches an HH convergence application (App), starts a listener (user datagram protocol (UDP), Universal plug and play (UPnP), etc.), displays a known image(e.g., simple) or series or set of images(e.g., complex) on the displayhaving a first characteristic (e.g., color, shade, pattern, shape, etc.), and the user holds the HH electronic deviceup to TV camerawith the displayfacing the TV camera. In one example embodiment, the image(s)ormay be: a colored blob, simple shape, or shape that may change over time; complex shape, static or dynamic pattern, etc.

310 311 120 140 310 311 141 120 140 140 310 311 120 122 In one example embodiment, in the case of the image(s)orbeing a static color blob or pattern, the TVsends broadcast commands to all HH electronic devices(e.g., connected with the network, e.g., a local area network (LAN), home network, etc.) to change the color of the image(s)oron respective displaysto a different but randomly selected characteristic (e.g., color, pattern, shape, etc.), e.g., (HH1→color 1, HH2→color 2 etc.). In one embodiment, a re-change request/command may be sent by the TVin the event that there are more than one HH electronic device, and the more than one HH electronic deviceshave used/chosen close a characteristic (e.g., colors, shapes, patterns, etc.) for the respective image(s)oras determined by the TVusing the TV camera.

311 310 311 310 140 311 310 140 In one or more embodiments, either a complex or simple color blob, shape or patternor a complex or simple series of color blobs, shapes or patternsare based on camera resolution, lightening, distance, etc. In one or more embodiments, if the camera or image sensor is able to recognize a complex color blob, shape or patternor series, then the number of color changes required to recognize an electronic devicemay be minimized. If a simple color blob, shape or patternor seriesneeds to be used, then a larger number of characteristic changes may be needed in order to recognize (or distinguish) an electronic device.

4 FIG. 400 140 120 320 411 141 140 120 140 140 411 140 411 140 140 shows another example scenariofor device pairing, according to an embodiment. In one embodiment, the HH electronic deviceheld in front of the TVby a userchanges imageobject characteristic (e.g., color, shade, shape, pattern, etc.) on the displayof HH electronic deviceto the one directed by the TVfor identifying which HH electronic deviceit is (among multiple HH electronic deviceswith other characteristics for the image). In one embodiment, the HH electronic devicechanges the characteristic by changing a pattern of an objectthat identifies which HH electronic device(of multiple HH electronic devices) it is.

5 FIG. 50 50 51 140 120 52 140 140 120 53 140 141 shows an example flow chart for a processfor device pairing, according to an embodiment. In one embodiment, the processstarts in block(e.g., an HH electronic deviceturns on, a TVturns on, etc.). In one embodiment, in blockan HH electronic device(a first electronic device) launches an application for pairing/associating the HH electronic devicewith a TV(second electronic device) in an environment or particular area, such as a home network, LAN, etc. In one embodiment, in block, the HH electronic devicedisplays an image(s) on a display (e.g., display) including a first characteristic (e.g., color, pattern, shape, etc.).

54 120 140 55 120 120 140 56 50 In one embodiment, in block, the TVattempts to recognize the image(s) and first characteristic for the electronic device. In one embodiment, in block, if the TVrecognizes the image and first characteristic, the TVand the electronic devicepair with one another. In one embodiment, blockthe pairing processstops, and the application may continue to another step, such as communicate particular messages with between the paired devices, share information between the paired devices, etc.

50 140 140 140 120 50 140 140 120 122 140 141 320 140 120 In one embodiment, processfurther includes if it is determined that the image and first characteristic is not recognizable or indistinguishable from any other electronic devicesin the particular environment or area (e.g., LAN, home network, room, etc.) the HH electronic device(and any other HH electronic devicesin the environment) receives a request to change the first characteristic to one or more second characteristic (e.g., color, shade, shape, pattern, etc.) from the TV. In one embodiment, processfurther includes the HH electronic device(and any other HH electronic devicesin the environment) changes the image to have the second characteristic. In one embodiment, a camera of the TV(TV camera) recognizes the change to the second characteristic by the HH electronic device, which has a displayplaced in front of the TV camera by a user (e.g., user) showing the second characteristic of the image. In one embodiment, the HH electronic device(first electronic device) is now paired with the TV(second electronic device).

6 FIG. 600 120 411 120 122 140 610 120 120 140 120 140 120 140 shows an example scenariofor device pairing, according to an embodiment. In one embodiment, after the HH electronic devicehas a characteristic of an imagerecognized by the TVusing the TV camera, the HH electronic devicebecomes pairedwith the TV. In one embodiment, the TVmay assign the HH electronic devicea unique image(s) (e.g., a unique blob) with a uniquely identifiable characteristic (e.g., color, shade, shape, pattern, etc.) for future use, other applications, etc. In one embodiment, the TVmay send an acknowledge message (ACK) to the HH electronic deviceonce the second characteristic of the image is recognized by the TV, and the HH electronic devicemay then continue with the next step on the application being executed or start another application.

7 FIG. 8 FIG. 8 FIG. 60 60 320 140 60 61 140 120 60 120 122 shows another example flow chart for a processfor device pairing, according to an embodiment. In one embodiment, processprovides for associating a user (e.g., user,) with a community/shared HH electronic device(). In one embodiment, processstarts in block, where the HH electronic device(first electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by the TVusing the TV cameraand the users have been assigned a name/Identification (ID).

62 120 140 63 64 140 65 120 140 In one embodiment, in blockthe TVperforms facial recognition for a user of a connected (to the network, e.g., home network, LAN, etc.) HH electronic device. In one embodiment, in blocka recognized user is assigned a distinct ID. In one embodiment, in blocka user places an HH electronic devicein front of the user's face. In one embodiment, in blockthe TVrecognizes the user face in front of the HH electronic device.

66 120 140 120 67 120 140 140 60 68 140 120 In one embodiment, in block, the TVdetermines the distance from the HH electronic deviceto the TV. In one embodiment, in blockthe TVassociates with the HH electronic deviceor associates with the user face of the HH electronic device. In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the HH electronic deviceand the TV, or communicate using another application, etc.

8 FIG. 700 320 700 120 140 700 140 120 140 120 700 122 122 141 140 700 122 120 120 shows an example scenariofor pairing with a userface, according to an embodiment. In one embodiment, for scenarioit is assumed that multiple TVsand multiple HH electronic devicesare present in the environment (e.g., a home environment). In one embodiment, for scenario, the HH electronic deviceand TVare on the same network (or AP), which may be a particular home setting and the HH electronic deviceand TVhave been paired. In one embodiment, it may be assumed for scenariothat the TV camerais constantly working (or may be started by a command, such as a voice command, RC command, etc.), and the TV camerais able to recognize a characteristic (e.g., color, pattern, shape, etc.) of an image or object (e.g., a blob, design, photograph, pattern, etc.) on the displayof the HH electronic device. In one embodiment, for scenarioit is assumed that the TV cameraand other processing by the TVis able to recognize and keep track of a user face (e.g., via facial recognition techniques). In one embodiment, it is also assumed that all users' faces have been identified by the TVand have been assigned a particular name/ID.

700 320 140 411 141 140 120 140 In one embodiment, in scenarioa userholds up the HH electronic deviceclose to their face and starts a convergence App. In one embodiment, the App displays an imagehaving a first characteristic (e.g., color, shape, pattern, etc.), such as a colored blob, on the displayof the HH electronic device. In one embodiment, the TVsends commands to all HH electronic devicesin the environment (e.g., in the home network, LAN, etc.) to change the first characteristic to a second characteristic (e.g., color, shape, pattern, etc.) that is a different, but randomly selected characteristic, e.g., color (HH1→color 1, HH2→color 2 etc.).

140 120 122 120 140 120 120 140 120 140 320 120 140 320 120 710 In one embodiment, all of the HH electronic deviceschange to a second characteristic, such as change of color of the image, as directed by the TV. In one embodiment, based on the new characteristic that the TV cameranow observes, the TVidentifies the HH electronic devicein front of the TV. In one embodiment, the TVrecognizes the characteristic change and sends a command to the HH electronic deviceto move to the next step in the App. In one embodiment, the TVrecognizes the face closest to the HH electronic device. Once the userface is recognized by the TV, the HH electronic deviceis associated to the face (or user), which is paired with the TVas shown by the arrow.

9 FIG. 800 810 320 120 800 120 140 800 140 120 140 120 800 122 122 141 140 800 122 120 120 shows an example scenariofor determining distancebetween paired devices, according to an embodiment. In one embodiment, determining the relative distance of the userwith respect to the TVmay be useful to determine display characteristics, such as the font size, personal preferences, etc. In one embodiment, for scenarioit is assumed that multiple TVsand multiple HH electronic devicesare present in the environment (e.g., a home environment). In one embodiment, for scenario, the HH electronic deviceand TVare on the same network (or AP), which may be a particular home setting and the HH electronic deviceand TVhave been paired. In one embodiment, it may be assumed for scenariothat the TV camerais constantly working (or may be started by a command, such as a voice command, RC command, etc.), and the TV camerais able to recognize a characteristic (e.g., color, pattern, shape, etc.) of an image or object (e.g., a blob, design, photograph, pattern, etc.) on the displayof the HH electronic device. In one embodiment, for scenarioit is assumed that the TV cameraand other processing by the TVis able to recognize and keep track of a user face (e.g., via facial recognition techniques). In one embodiment, it is also assumed that all users' faces have been identified by the TVand have been assigned a particular name/ID.

800 320 140 140 120 140 In one embodiment, in scenarioa userholds up the HH electronic deviceclose to their face and starts the convergence App. In one embodiment, the App shows a fixed size image (e.g., an object, a blob, etc.) with a first characteristic (e.g., color, shape, pattern, etc.), which is associated with the HH electronic device. In one embodiment, the TVsends commands to all HH electronic devices(connected in the environment) to change the first characteristic to a second characteristic (e.g., color, shape, pattern, etc.), which is different from the first characteristic but randomly selected, e.g., color (HH1→color 1, HH2→color 2 etc.).

140 120 411 120 120 120 140 140 141 120 810 411 122 In one embodiment, the HH electronic devicein front of TVchanges the imageto the second characteristic (e.g., blob color) to the second characteristic directed by the TVfor identifying which HH electronic deviceit is. In one embodiment, the TVrecognizes the characteristic change and sends a command to the HH electronic deviceto move to the next App step. In one embodiment, since the HH electronic deviceshows a fixed size image with the second characteristic on the display, the TVmay determine the distancebased on the size of the imageas seen by TV camera.

810 140 120 141 140 122 122 320 140 120 140 320 320 140 120 140 140 In one example embodiment, the distanceof the HH electronic devicefrom the TVmay be equal to the diameter of the image (e.g., circle (blob)) shown on the displayof the HH electronic device, divided by the diameter of the image (e.g., a circle) seen by the TV cameramultiplied by the focal length of the TV cameralens (e.g., Distance=Camera focal length*(diameter of the HH circle/diameter of the image circle)). In one embodiment, if the useris holding a personal HH electronic device, the TVis now paired to the HH electronic device(and therefore, paired to the user). In one embodiment, if the useris holding a personal communal HH electronic device, the TVrecognizes the face closest to the HH electronic device, and from there on, the HH electronic deviceis associated to the particular closest face (or user).

10 FIG. 70 70 71 140 120 70 120 122 shows another example flowchart of a processfor determining distance between paired devices, according to an embodiment. In one embodiment, processstarts in block, where the HH electronic device(first electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by the TVusing the TV cameraand the users have been assigned a name/Identification (ID).

72 120 140 73 74 140 75 120 140 In one embodiment, in blockthe TVperforms facial recognition for a user of a connected (to the network, e.g., home network, LAN, etc.) HH electronic device. In one embodiment, in blocka recognized user is assigned a distinct ID. In one embodiment, in blocka user places an HH electronic devicein front of the user's face. In one embodiment, in blockthe TVrecognizes the user face in front of the HH electronic device.

76 120 140 140 120 77 140 140 140 140 70 78 140 120 In one embodiment, in block, the TVdetermines the distance from the first HH electronic deviceto a second HH electronic device, where both are in front of the TV. In one embodiment, in blockthe first HH electronic deviceis associated with the second HH electronic device, or the user face of the first electronic deviceis associated with a second user face of the second HH electronic device. In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the first and second HH electronic deviceand the TV, or communicate using another application, etc.

11 FIG. 900 900 320 920 900 900 120 140 900 140 120 140 120 900 122 122 141 140 900 122 120 120 900 320 920 140 140 shows example scenariofor device pairing and device relative location determination and tracking, according to an embodiment. In one embodiment, the scenariodetermines the relative position of two (or more) persons (e.g., userand user). In one embodiment, scenariois useful for multiplayer games where relative position of players is important (example car racing, card games, etc.). In one embodiment, for scenarioit is assumed that multiple TVsand multiple HH electronic devicesare present in the environment (e.g., a home environment). In one embodiment, for scenario, the HH electronic deviceand TVare on the same network (or AP), which may be a particular home setting and the HH electronic deviceand TVhave been paired. In one embodiment, it may be assumed for scenariothat the TV camerais constantly working (or may be started by a command, such as a voice command, RC command, etc.), and the TV camerais able to recognize a characteristic (e.g., color, pattern, shape, etc.) of an image or object (e.g., a blob, design, photograph, pattern, etc.) on the displayof the HH electronic device. In one embodiment, for scenarioit is assumed that the TV cameraand other processing by the TVis able to recognize and keep track of a user face (e.g., via facial recognition techniques). In one embodiment, it is also assumed that all users' faces have been identified by the TVand have been assigned a particular name/ID. In one embodiment, in scenariois may be assumed that the players (e.g., usersand) are using communal HH electronic devicedevice (e.g., a game controller) or a personal HH electronic device(e.g., personal game controller).

900 800 320 920 140 930 940 140 320 920 120 122 320 920 120 122 320 920 120 320 920 320 920 In one embodiment, scenariois an extension of scenariowhere each userandis paired with a specific HH electronic device, as indicated by the arrowsand. In one example embodiment, once the two HH electronic devicesare paired with one another (or the user'sandfaces are associated), then the TVmay follow each user by using the TV cameraand keep track of the usersandrelative location. In one embodiment, the TVusing the TV cameradetermines the relative distance between the two usersand, and content shown on the TVmay be tailored to the usersandbased on their particular location relative to one another, such as personal information related to a game, content related to a program or application, content based on personal profiles of each userand, etc.

12 FIG. 80 80 81 140 140 120 80 120 122 shows another example flowchart of a processfor device pairing and device relative location determination and tracking, according to an embodiment. In one embodiment, processstarts in block, where an HH electronic device(first electronic device) and another HH electronic device(third electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by the TVusing the TV cameraand the users have been assigned a name/ID.

82 120 140 83 84 140 140 140 85 120 140 In one embodiment, in blockthe TVperforms facial recognition for a user of the first and second connected (to the network, e.g., home network, LAN, etc.) HH electronic devices(first and third electronic devices). In one embodiment, in blockthe recognized users of the two HH electronic devices are assigned distinct IDs. In one embodiment, in blocka user of the first HH electronic device(first electronic device) and the user of the second HH electronic device(third electronic device) places the respective HH electronic devicein front of the respective user face. In one embodiment, in blockthe TVrecognizes the user's faces in front of each HH electronic device.

86 140 140 87 120 140 140 120 88 120 122 140 140 80 89 140 120 In one embodiment, in block, the first user face is associated with the first HH electronic device(first electronic device) and the second user face is associated with the second HH electronic device(third electronic device). In one embodiment, in blockthe TVdetermines the relative position between the first HH electronic deviceand the second HH electronic device, where both are in front of the TV. In one embodiment, in blockthe TVusing the TV camerafollows movement of the first HH electronic deviceand the second HH electronic device. In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the first and second HH electronic deviceand the TV, or communicate using another application, etc.

13 FIG. 1000 140 120 140 320 920 120 122 140 shows an example timing diagramfor device sharing and pairing, according to an embodiment. In one embodiment, to easily access a Wi-Fi network from a HH electronic devicein a semi-trusted domain (e.g., friend's residence, organization environment, etc.), where the host does not want to give their AP service set identifier (SSID) and password to the guest. One or more embodiments make it easy for the guest if they want to connect to a TVfor a convergence App (e.g., a multi-player gaming app, etc. One or more embodiments determines the relative position of the HH electronic deviceuser (e.g., user/) with respect to the TVby using the TV camera, which may be used to determine relative location of multiple users or for pointing to a device using the HH electronic device.

140 120 120 120 140 120 140 120 122 141 140 122 120 320 920 120 11 FIG. In one embodiment, a guest in the semi-trusted environment using an HH electronic devicedoes not know the host AP SSID and password. In one embodiment, there are multiple TVsin the host home (note that if there is only one TV, selecting a TVfrom a list becomes easier). In one embodiment, there are multiple guests with HH electronic devicesat the host home. In one embodiment, the host TVand guest HH electronic deviceare not been paired. In one embodiment, the TVuses the TV camerafor recognizing an image on a displayof HH electronic deviceswhen held in front of the TV camera. In one embodiment, the TVmay identify a user/() and assign the user an ID. In one embodiment, the TVmay recognize and match a previously identified user.

120 140 320 920 120 140 140 11 FIG. In one embodiment, once the TVrecognizes the users of HH electronic devices(e.g., userand user,) based on image and characteristic recognition and/or facial recognition, the TVsends the AP SSID and password to the HH electronic device(s), which may be able to share the AP SSID with other HH electronic devicesif permitted by the host.

1000 140 140 120 120 140 141 122 In one example embodiment, using the timing diagram, a guest HH electronic deviceis provided with the host AP SSID and password. In one example, assuming that it is desired to launch a convergence App, the guest HH electronic devicealso desires to pair with host TV. In one embodiment, the Host device (e.g., TV) launches a TV convergence App (e.g., a shareable App). In one embodiment, the guest launches an HH electronic deviceconvergence App, which shows a known image or series of images with a first characteristic (e.g., color, shape, pattern, etc.), and the user holds the HH electronic device with the displayfacing the TV camera.

120 130 122 140 141 120 140 2 FIG. In one embodiment, the TVrecognizes the image and first characteristic, and sends a time stamped command (e.g., “New HH found”) to a server (e.g., server,) with the captured image as seen by the TV camera. In one embodiment, the HH electronic devicesends a time stamped command (e.g., “Showing”) to the same server with what it displayed on the display. In one embodiment, the server matches the TVand the HH electronic devicebased on the time stamp and displayed image/objects.

120 140 120 140 140 120 140 In one embodiment, it is possible that there is more than one TVor HH electronic devicethat is matched (e.g., more than 2 commands sent in a time window). In one embodiment, there are multiple options to narrow down the list of potential matches and reduce potential false positives. In one embodiment, both the TVand the HH electronic devicesend out their respective location (or approximation thereof) using, for example, GPS, known information, text information of a known location, etc., which narrows down the search space. In one embodiment, assuming that the host sends out an invitation to all the guests (e.g., email, text message, etc.), the “party” information (e.g., email addresses) may be stored in the server. In one example embodiment, when the “New HH found” and “Showing” commands reach the server, the server may pair HH electronic devicesbased on the party (host TVand guest HH electronic device).

140 141 120 In one example embodiment, each HH electronic deviceshows a random selected image with a characteristic (e.g., a color, shape, pattern, etc.) on a respective display. In one embodiment, the TVrecognizes the image and characteristic, and informs the server regarding the image and characteristic.

140 120 120 140 In one example embodiment, the wireless APs in the vicinity of the HH electronic devicesmay be matched against the ones the TVis distributing credentials for using the SSID, BSSID or other uniquely identifying information along with signal strength. The one or more example embodiments assist in reducing the list of potential TV←→HH electronic devicepairing options.

120 140 120 140 122 In one or more embodiments, to resolve remaining multiple matches the following may be done. In one example embodiment, the TVsends commands to all HH electronic devices(via the server) that are candidates for the pairing. In one example embodiment, the “Change” command is used to change a characteristic (e.g., color, shape, pattern, etc.) to a different but randomly selected characteristic (e.g., HH1→color 1, HH2→color 2 etc.). In one embodiment, the TVrecognizes the change of the characteristic for the HH electronic devicein front of the TV cameraand sends this information to the server.

140 140 120 120 122 140 In one embodiment, the HH electronic devicenarrows its location, and then requests from the server a unique code (e.g., a simple or complex: color blob, shape, pattern, etc. with a characteristic (e.g., color, shade, etc.)) to display for this subset of devices it must also identify. In one example embodiment, the HH electronic devicethen displays code to the TV. In one example embodiment, the TVthen sends to the server what the TV cameracaptured from the HH electronic device.

120 120 120 140 140 120 120 140 120 140 140 140 120 In one example embodiment, the TVthen sends the AP SSID and password to the server along with its name, which may be set in the App or via the TV. In one embodiment, the AP SSID and password are auto-populated if the TVis wirelessly connected. In one embodiment, the server identifies the HH electronic devicebased on the color reported by the TV and sends the AP SSID, password, and TV name to it. In one example embodiment, the HH electronic deviceapplies these AP settings for connecting with the TVfor sharing the App (e.g., Party App, etc.). In one example embodiment, the AP settings information is encrypted so that the user cannot access it. In one example embodiment, to confirm that Wi-Fi settings were correctly applied, there may be another communication session between the TVand the HH electronic deviceover Wi-Fi. In one example embodiment, the TVsends an ACK to the HH electronic deviceand the HH electronic devicenow continues with the next step on the App. In one example embodiment, the GPS on the HH electronic devicemay be used to determine the TVslocation for future use.

120 140 140 140 120 140 120 140 140 120 122 140 120 140 120 140 120 120 In one embodiment, the TVis able to select the HH electronic devicein front of it from other HH electronic devicesthat might be eligible and vice versa (i.e. the HH electronic devicemay select the TVwhich it is in front of) depending on which device initiates the discovery. One or more embodiments fall into three categories: Time multiplexed: The image characteristic on the HH electronic devicechanges in response to the TV(or server) command and the search space is reduced progressively. In one example embodiment, this approach allows for low fidelity camera and simple image detection algorithms, but takes more time since the characteristic may have to change multiple times before the HH electronic deviceis uniquely identified; Code multiplexed: The image that is shown on the HH electronic deviceis made up of complex shapes and various characteristics (e.g., colors) which carries more information and makes it more likely to be unique. In one embodiment, this approach forces the TVto have a high-fidelity TV cameraand more complex image detection algorithms but reduces the time it takes to uniquely identify the HH; Location multiplex: These approaches limit the number of HH electronic devicesthat may connect to a TV(reduce the search space). This may be achieved by: Using GPS so that the HH electronic deviceonly searches for a TVin the neighborhood; or Using an invite from a party host to limit only the guest HH electronic deviceto connect to the party host TV. In one embodiment, these techniques may be used individually or in conjunction with each other to solve the search space reduction problem. In one example embodiment, the technique used depends on the camera fidelity and algorithm technology implemented on the TV.

140 120 120 140 In one example embodiment, the relative distance of the user of an HH electronic devicewith respect to the TVis determined. In one embodiment, this determination is useful to determine the font size, personal preferences, etc. In one example embodiment, all users' faces have been identified by the TVand they have been assigned a name/ID. In one embodiment, the HH electronic devicesmay be paired with the one or more embodiments described above.

140 120 140 140 In one embodiment, the relative position of two (or more) persons based on the HH electronic devicesmay be determined. In one example embodiment, this is useful for multiplayer games where relative position of players is important (e.g., car racing video games, card games, etc.). In one example embodiment, all user faces have been identified by the TVand they have been assigned a name/ID. In one example embodiment, the players are using a communal HH electronic device(e.g., a communal game controller) or a personal HH device electronic device(e.g., a personal game controller). In one example embodiment, this scenario may be implemented in the same way as described above in one or more embodiments.

14 FIG. 90 90 91 140 120 90 120 122 shows another example flowchart for a processdevice sharing and pairing, according to an embodiment. In one embodiment, processstarts in block, where an HH electronic device(first electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by a TVusing a TV cameraand the users have been assigned a name/ID.

92 120 130 140 141 122 93 140 141 94 120 120 140 2 FIG. In one embodiment, in blockthe TVsends a time-stamped command to a server (e.g., server,) including an image with a first characteristic (e.g., color, shape, pattern, etc.) that was captured from an HH electronic devicedisplayusing a TV camera. In one embodiment, in blockthe HH electronic devicesends a time-stamped command to the server including the image displayed using the display. In one example embodiment, in block, the server pairs the HH electronic device with the TVbased on matching the received images and the commands including time-stamps from the TVand the HH electronic device.

95 120 140 96 120 140 122 97 140 140 140 In one example embodiment, in blockthe TVrecognizes a characteristic change to the image displayed by the HH electronic device. In one example embodiment, in blockthe TVsends the server information about the image and changed characteristic as captured from the HH electronic deviceusing the TV camera. In one example embodiment, in blockthe HH electronic deviceprovides information assisting in narrowing location determination (e.g., GPS, text including location information, etc.) of the HH electronic deviceto the server and requests a unique code (e.g., simple or complex: color blob, shape, pattern, etc., with a characteristic (e.g., color, shade, etc.)) to display for any devices that the HH electronic deviceneeds to identify.

98 140 120 122 120 90 99 140 120 In one example embodiment, in blockthe HH electronic devicereceives the unique code and displays the unique code in front of the TV, which uses the TV camerato capture the unique code, where the TVsends the unique code to the server. In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the HH electronic deviceand the TV, or communicate using another application, etc.

15 FIG. 1500 1500 1501 140 120 1500 120 122 shows another example flowchart for a processfor device pairing, determining distance between the paired devices and adjusting a display based on the pairing, according to an embodiment. In one embodiment, processstarts in block, where an HH electronic device(first electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by a TVusing a TV cameraand the users have been assigned a name/ID.

1502 120 130 140 141 122 1503 140 141 1504 120 140 2 FIG. In one embodiment, in blockthe TVsends a time-stamped command to a server (e.g., server,) including an image with a first characteristic (e.g., color, shape, pattern, etc.) that was captured from an HH electronic devicedisplayusing a TV camera. In one embodiment, in blockthe HH electronic devicesends a time-stamped command to the server including the image displayed using the display. In one example embodiment, in block, the TVrecognizes a user face of the HH electronic device.

1505 120 140 120 141 140 122 122 1506 120 140 120 In one example embodiment, in blockthe TVdetermines the distance from the HH electronic deviceto the TVbased on a size of the image displayed on the displayof the HH electronic device, the size of the captured image by the TV cameraand the focal length of the TV camera. In one example embodiment, in block, the display of the TVis adjusted based on one or more personal preferences associated with the user face of the HH electronic deviceand the determined distance between the HH electronic device and the TV.

1500 1507 140 120 In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the HH electronic deviceand the TV, or communicate using another application, etc.

16 FIG. 1600 120 1600 1601 140 140 120 1600 120 122 shows another example flowchart for a processfor device pairing, determining distance between multiple paired devices and adjusting a display of a TVbased on the pairing, according to an embodiment. In one embodiment, processstarts in block, where a first HH electronic device(first electronic device) and a second HH electronic device(third electronic device) turns on, starts an application, etc. and the TV(second electronic device) turns on, starts an application, etc.). In one embodiment, for processit may be assumed that all users' faces have been identified by a TVusing a TV cameraand the users have been assigned a name/ID.

1602 120 130 140 141 122 1603 140 141 1604 120 140 140 2 FIG. In one embodiment, in blockthe TVsends a time-stamped command to a server (e.g., server,) including images with first characteristics (e.g., color, shape, pattern, etc.) that were captured from the first and second HH electronic devicesdisplaysusing a TV camera. In one embodiment, in blockthe first and second HH electronic devicessend time-stamped commands to the server including the respective images displayed using the respective displays. In one example embodiment, in block, the TVperforms facial recognition for recognizing user faces of the connected HH electronic devices(e.g., the first and second HH electronic devices).

1605 140 140 140 120 140 In one example embodiment, in blockthe user of the first HH electronic deviceand the user of the second HH electronic deviceplace the respective HH electronic devicesin front of the respective user faces. In one embodiment, the TVrecognizes the faces of each user of each HH electronic device.

1606 120 140 140 1607 120 140 In one embodiment, in blockthe TVdetermines the relative position (similarly as for the one or more embodiments described above) of the first HH electronic deviceas compared with the second HH electronic device. In one example embodiment, in block, the display of the TVis adjusted based on the determined relative position of the first and second HH electronic devices.

1600 1608 140 120 In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the first and second HH electronic devicesand the TV, or communicate using another application, etc.

17 FIG. 1700 140 120 122 140 shows another example flowchart for a processfor facial pairing, determining distance between paired devices and adjusting a display based on the pairing, according to an embodiment. One or more embodiments provide for easy pairing of a personal HH electronic devicewith a communal fixed device (TVand TV camera) in a Campus Area Network (CAN) or Metropolitan Area Network (MAN) to exchange information. One or more embodiments, easily and accurately identify a location of an HH electronic devicein a closed space where GPS, Wi-Fi and other location identification methods may not work accurately (e.g., due to reception issues, interference, jamming, etc.).

140 140 140 122 In one or more embodiments, HH electronic devicesare paired in an un-trusted CAN/MAN Wi-Fi network (e.g., airports, schools, malls, etc.) for accurately identifying location. It is noted that pairing is not an issue if AP SSID and password are available (or can be guessed), but accurately identifying the location of an HH electronic devicein the absence of specific information (e.g., GPS information) may be an issue. One or more embodiments locate the HH electronic devicewithin line-of-sight and in the neighborhood of a TV camerathat is mounted on a TV (or computer), or just buy itself.

140 In one or more embodiments, incorporating local pairing, or semi-trusted domain pairing embodiments as described above, a user is provided the same user experience for pairing devices regardless of where they are situated (e.g., home, home of a friend, public place, etc.). In one example embodiment, devices are paired and position of an HH electronic deviceis accurately located without having to be in the physical proximity of a fixed communal device (e.g., NFC reader, QR code reader, etc.). One or more embodiments provide for personalized (or changing) information to be shared between devices, which is not possible with NFC or QR codes.

140 120 140 140 120 140 122 140 In one or more embodiments, the CAN or MAN environment (e.g., mall, airport, colleges, etc.) AP with or without repeater and TV (or camera) owner/controller allows a user of an HH electronic deviceto be on the same network, which may be typically true (e.g., airport, malls, hotels, college campus, etc.). In one example embodiment, multiple TVsand multiple HH electronic devicesexist in an un-trusted CAN/MAN domain. In one example embodiment, the user of an HH electronic devicedoes not know a host AP SSID and password (even though it might be easily available). In one example embodiment, the host TVand guest HH electronic devicehave not been paired. In one example embodiment, it is assumed that the TV camerais constantly working (or may be started by a command) and is able to recognize an image with a characteristic (e.g., color, shape, pattern, etc.) on the HH (electronic device.

140 120 1700 1701 140 120 In one or more embodiments, a user of an HH electronic deviceis provided with a CAN/MAN AP SSID and password. In one embodiment, the user desires to launch a convergence App, and also desires to pair with CAN/MAN TV. In one embodiment, the processstarts in block, where an HH electronic device(first electronic device) turns on, starts an application (e.g., convergence App), etc. and the TV(second electronic device) turns on, starts an application, etc.).

1702 120 130 140 141 122 1703 140 141 1704 140 120 2 FIG. In one embodiment, in blockthe TVsends a time-stamped command to a server (e.g., server,) including an image with a first characteristic (e.g., color, shape, pattern, etc.) that was captured from the HH electronic devicedisplayusing a TV camera. In one embodiment, in blockthe HH electronic devicesends a time-stamped command to the server including the image displayed using the display. In one embodiment, in blockif location information is available (e.g., GPS signals, Wi-Fi, etc.), the HH electronic deviceand TVsend the location information to the server.

1705 140 120 140 120 140 120 In one embodiment, in blockthe server matches the location information of the HH electronic deviceand the TVusing the location information received from each device. In one embodiment, multiple HH electronic devicesand multiple TVssend their respective location information, which is then matched by the server to determine which HH electronic deviceis in front or near to which TV.

1706 140 120 1707 120 140 120 120 140 In one embodiment, in block, the HH electronic device(first electronic device) is placed in front of the user's face, and the TV(second electronic device) recognizes the user face (e.g., performs facial recognition). In one embodiment, in blockthe TVassociates the user face (first user face) with the HH electronic device. In one embodiment, the server pairs the HH electronic device with the TVbased on matching the received images and the commands including time-stamps from the TVand the HH electronic device, and the location information if available.

1708 120 140 120 140 122 120 122 1709 120 140 1710 120 140 120 In one embodiment, in blockthe TVdetermines the distance from the HH electronic deviceto the TVbased on the size of the displayed image on the HH electronic device, the size of the image captured by the TV cameraof the TV, and the focal length of the TV camera. In one example embodiment, in blockthe TVmay determine the relative position of the HH electronic devicewith respect to the user face. In one embodiment, in blockthe TVadjusts the display based on one or more of the determined distance to the HH electronic deviceand the determined relative position of the user face. In one embodiment, the TVadjusts the display to the user of the HH electronic device by targeting text/graphics/video in a particular position on the screen, increases font size, etc.

1700 1711 140 120 In one embodiment, the processstops at block, where the application may proceed to a next step for communication between the HH electronic deviceand the TV, or communicate using another application, etc.

140 120 122 120 1700 140 140 One embodiment provides for precisely locating the position of a user HH electronic devicein an enclosed space (mall, airport, college) etc. where GPS signals or Wi-Fi might not be accurate (e.g., location information is not available). In one embodiment, the precise location of the user may be determined relative to the location of the TV(or TV camera). In one embodiment, the TVis static and its position is known (e.g., by a server) and it does not change. In one embodiment, similarly as to process, the user holds up an HH electronic deviceand starts the convergence App. In one example embodiment, the App shows a fixed size image with a characteristic (e.g., color, shape, pattern, etc.) which is associated with the HH electronic device.

1700 140 120 122 140 120 122 122 In one example embodiment, the TV-HH pairing occurs as in process. In one embodiment, since the HH electronic deviceshows a fixed size image with a characteristic, the TVmay determine the distance based on the size of the image as seen by TV camera. In one embodiment, the distance of the HH electronic devicefrom TVis equal to the diameter of the image (e.g., a circle, blob, etc.) divided by the diameter of the image received by the TV cameramultiplied by the focal length of the TV cameralens.

120 140 120 140 122 In one example embodiment, the server may request the HH electronic devices to change the characteristic of the displayed image, and the TVrecognizes a characteristic change to the image displayed by the HH electronic device. In one example embodiment, the TVsends the server information about the image and changed characteristic as captured from the HH electronic deviceusing the TV camera.

120 120 120 140 120 In one or more embodiments, in the CAN/MAN environment, various uses may be applied. In one example, in a sports bar, persons in front of the TVmay be identified and based on their identity (and associated profile information, personal information known, etc.), the TVshows the user particular scores of preferred teams or interested games/events/etc. only on a particular TVbased on the user of the HH electronic devicebeing in front of or near to the particular TV.

140 120 120 140 120 140 In one example embodiment, users of HH electronic devicesmay sit in front of a TVand request the TVto show only information that they are interested in from commands from their HH electronic devices. In one embodiment, the TVmay arbitrate this information if there are multiple HH electronic devices(e.g., majority requests, switching between requests using weighted time periods, etc.).

140 120 120 In one example embodiment, in an airport, the precise location of the person using an HH electronic devicemay be identified by the TVthat they are standing in front of. In one example embodiment, the TVmay provide the particular users directions, arrival/destination information pertinent to them, or provide live assistance.

120 120 120 140 In one example embodiment, in a mall, the TVmay show information and photos/video (e.g., with products, services, etc.) on a TVnear the user and make it a social experience. In one embodiment, the TVmay provide for users of HH electronic deviceto play games, be offered coupons that tie a consumer to a location, etc.

120 140 140 120 140 140 120 In one embodiment, multiple TVsthat are positioned in series or in the path direction of a user of an HH electronic devicemay transition based on the position of the HH electronic device. In one embodiment, the user may be targeted with content on a specific TVbased on the time the HH electronic deviceremains in front of a specific place (e.g., store, display, etc.). In one embodiment, based on information obtained for multiple users of HH electronic devicesthat may be traveling together, the TVmay determine common information that the multiple users may be interested in to target both users simultaneously.

18 FIG. 1800 1800 1840 1830 1820 120 1810 122 shows another exampledetermination of location of a user relative to a perpendicular from a paired device, according to an embodiment. In one embodiment, exampleshows the user position (angle) relative to the perpendicularfrom the face of the TV. In one embodiment, a user who is located at a distance “d”from the center of the image. The angle “a”that the user makes with the perpendicular from the face of the TVis given by the simple formulaea=arc tan(d/f)where “f”is the focal length of the TV camera.

19 FIG. 500 500 511 512 513 514 515 516 517 517 550 551 552 553 500 518 511 517 is a high-level block diagram showing an information processing system comprising a computing systemimplementing an embodiment. The systemincludes one or more processors(e.g., ASIC, CPU, etc.), and can further include an electronic display device(for displaying graphics, text, and other data), a main memory(e.g., random access memory (RAM)), storage device(e.g., hard disk drive), removable storage device(e.g., removable storage drive, removable memory module, a magnetic tape drive, optical disk drive, computer-readable medium having stored therein computer software and/or data), user interface device(e.g., keyboard, touch screen, keypad, pointing device), and a communication interface(e.g., modem, wireless transceiver (such as WiFi, Cellular), a network interface (such as an Ethernet card), a communications port, or a PCMCIA slot and card). The communication interfaceallows software and data to be transferred between the computer system and external devices through the Internet, mobile electronic device, a server, a network, etc. The systemfurther includes a communications infrastructure(e.g., a communications bus, cross-over bar, or network) to which the aforementioned devices/modulesthroughare connected.

517 517 The information transferred via communications interfacemay be in the form of signals such as electronic, electromagnetic, optical, or other signals capable of being received by communications interface, via a communication link that carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, a radio frequency (RF) link, and/or other communication channels.

500 520 145 519 500 521 522 523 524 525 526 527 2 FIG. In one implementation of one or more embodiments in a mobile wireless device such as a mobile phone, the systemfurther includes an image capture device, such as a camera(), and an audio capture device, such as a microphone. The systemmay further include application modules as MMS module, SMS module, email module, social network interface (SNI) module, audio/video (AV) player, web browser, image capture module, etc.

500 530 530 529 500 The systemfurther includes a network access (and pairing) and application connection moduleas described herein, according to an embodiment. In one implementation of network access and application connection processesalong with an operating systemmay be implemented as executable code residing in a memory of the system. In another embodiment, such modules are in firmware, etc.

As is known to those skilled in the art, the aforementioned example architectures described above, according to said architectures, can be implemented in many ways, such as program instructions for execution by a processor, as software modules, microcode, as computer program product on computer readable media, as analog/logic circuits, as application specific integrated circuits, as firmware, as consumer electronic devices, AV devices, wireless/wired transmitters, wireless/wired receivers, networks, multi-media devices, etc. Further, embodiments of said Architecture can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements.

Embodiments have been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to one or more embodiments. Each block of such illustrations/diagrams, or combinations thereof, can be implemented by computer program instructions. The computer program instructions when provided to a processor produce a machine, such that the instructions, which execute via the processor, create means for implementing the functions/operations specified in the flowchart and/or block diagram. Each block in the flowchart/block diagrams may represent a hardware and/or software module or logic, implementing one or more embodiments. In alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures, concurrently, etc.

The terms “computer program medium,” “computer usable medium,” “computer readable medium”, and “computer program product,” are used to generally refer to media such as main memory, secondary memory, removable storage drive, a hard disk installed in hard disk drive. These computer program products are means for providing software to the computer system. The computer readable medium allows the computer system to read data, instructions, messages or message packets, and other computer readable information from the computer readable medium. The computer readable medium, for example, may include non-volatile memory, such as a floppy disk, ROM, flash memory, disk drive memory, a CD-ROM, and other permanent storage. It is useful, for example, for transporting information, such as data and computer instructions, between computer systems. Computer program instructions may be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

Computer program instructions representing the block diagram and/or flowcharts herein may be loaded onto a computer, programmable data processing apparatus, or processing devices to cause a series of operations performed thereon to produce a computer implemented process. Computer programs (i.e., computer control logic) are stored in main memory and/or secondary memory. Computer programs may also be received via a communications interface. Such computer programs, when executed, enable the computer system to perform the features of one or more embodiments as discussed herein. In particular, the computer programs, when executed, enable the processor and/or multi-core processor to perform the features of the computer system. Such computer programs represent controllers of the computer system. A computer program product comprises a tangible storage medium readable by a computer system and storing instructions for execution by the computer system for performing a method of one or more embodiments.

Though the embodiments have been described with reference to certain versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.