Systems and Methods for Equalizing Audio for Playback on an Electronic Device

This application is a continuation of U.S. application Ser. No. 19/029,119, filed Jan. 17, 2025, which is a continuation of U.S. application Ser. No. 18/206,021, filed Jun. 5, 2023, issued as U.S. Pat. No. 12,238,488 on Feb. 25, 2025, which is a continuation of U.S. application Ser. No. 16/299,050, filed Mar. 11, 2019, issued as U.S. Pat. No. 11,706,577 on Jul. 18, 2023, which is a continuation of U.S. application Ser. No. 15/818,444, filed Nov. 20, 2017, issued as U.S. Pat. No. 10,405,113 on Sep. 3, 2019, which is a continuation of U.S. application Ser. No. 15/341,798, filed Nov. 2, 2016, issued as U.S. Pat. No. 9,854,374 on Dec. 26, 2017, which is a continuation of U.S. application Ser. No. 14/464,789, filed Aug. 21, 2014, issued as U.S. Pat. No. 9,521,497 on Dec. 13, 2016, all of which are hereby incorporated by reference in their entireties.

This application generally relates to improving audio playback on electronic devices. In particular, the application relates to platforms and techniques for applying equalization settings to audio data to be output by an electronic device based on a position or orientation of an active speaker of the electronic device relative to a surface.

Electronic devices such as smart phones support various channels and components for audio playback. For example, a user of an electronic device may participate in a telephone call by listening via an “earpiece” or “speakerphone” speaker of the electronic device. Further, the electronic device may output music via one or more built-in speakers. Additionally, the user may leverage an external speaker connected to the electronic device for added or enhanced audio playback.

There are various existing techniques to process audio data that is output via the speaker components of the electronic devices. In some existing devices, processing logic modifies the incoming signal to a speaker via various audio signal processing techniques based on volume settings, frequency response feedback, pressure feedback, or impedance feedback. In other devices, the orientation of peripheral headphones connected to the device is used to map sound signals that are provided to the headphones.

There is an opportunity to use data from one or more sensors of an electronic device to process audio data that is to be output by the electronic device.

Audio equalization is a technique used to alter the frequency and/or phase and/or time response of audio data, such as via the application of analog or digital filters. For example, filters may be applied to audio data to adjust the bass and treble tones present in outputted audio. Existing electronic devices support various applications that utilize speaker components to output audio, such as telephone applications, music applications, video conferencing applications, video players, social media applications, navigation applications, and others. Techniques to equalize the audio data that is output during operation of these various applications can lead to an improved user experience.

The embodiments described here process data from various built-in sensors of an electronic device to determine the general environment and orientation of the electronic device, and specifically an active speaker and speaker grille, relative to supporting or nearby surfaces and optionally to the listener. Generally, the orientation of the electronic device, including the orientation of the active speaker and its output grille, has an effect on the overall quality and sound of outputted audio. For example, if a built-in speaker grille of an electronic device is oriented flat against a surface, the audio output from a speaker may not have the intended bass tones, treble tones, resonance, and/or other audio characteristics. Typical electronic devices are equipped with various sensors whose data can indicate orientations and positions of the electronic devices. In particular, the sensors may include imaging sensors (e.g., cameras), proximity sensors, accelerometers, gyroscopes, location modules, ultrasonic sensors, infrared sensors, and others.

The electronic devices can store and adapt equalization settings that correspond to various local positions, orientations, or general environments for the electronic devices, whereby each of the equalization settings incorporates filters configured to improve or optimize the audio output for that particular local position of the electronic device relative to supporting surfaces and, in some cases, to the listener. For example, an electronic device may be stable and stationary with its built-in speaker grille(s) positioned flat (although possibly also recessed) against a surface. For further example, the electronic device may be stable and stationary with its built-in speaker(s) exposed but orientated away from a user. Further, for example, the electronic device may be stable but not stationary when it is traveling in a vehicle and secured in a dock or mount. Generally, an electronic device may be considered “stable” when there is little to no movement of the electronic device relative to a support surface(s). Further, an electronic device may be considered “stationary” when it is not moving. Accordingly, an electronic device may have absolute stability when (1) the electronic device itself is not moving (i.e., stationary) and (2) there is little to no movement of the electronic device relative to a surface(s); or relative stability when (1) the electronic device is moving (i.e., not stationary) and (2) there is little to no movement of the electronic device relative to a surface(s) (e.g., if the electronic device is mounted to a mount in a vehicle).

According to embodiments, the electronic device may collect data from various sensors and analyze the data to determine the local position of the electronic device. Further, the electronic device may identify a stored equalization setting that corresponds to the determined local position of the electronic device. In some optional embodiments, the electronic device may modify the equalization setting based on various factors such as acoustic input data (e.g., audio data generated by a microphone), optical data (e.g., data indicating whether a user is facing the speaker), infrared data, ultrasonic data, or other data. The electronic device may then apply the modified or unmodified equalization setting to the audio data, and output the equalized audio data via one or more speakers. In some cases, different speakers may have different equalization settings. The embodiments as discussed herein offer a benefit to users of the electronic devices by providing an improved audio playback experience. This benefit is especially important as electronic devices become more advanced and more incorporated into everyday use.

depicts a front view of an example electronic deviceconfigured to facilitate audio equalization processing and audio output. The electronic devicemay be any type of portable electronic device, for example, a notebook computer, a mobile phone, a Personal Digital Assistant (PDA), a smart phone, a tablet computer, a multimedia player, an MP3 or MP4 player, a digital or analog broadcast receiver, a remote controller, or any other electronic apparatus. It should be appreciated that the front side of the electronic devicecan be of various shapes and sizes. For example, the front side of the electronic devicecan be flat, curved, angled, flexible, or the like.

The electronic devicecan include audio components such as a speakerwith a grille, and a microphonewith an aperture. The speakeris configured to output audio based on an electrical audio signal and the microphoneis configured to convert detected sound into an electrical signal. As illustrated in, the speakeris an “earpiece” speaker that is commonly utilized by a user during telephone calls or similar applications. It should be appreciated that the types, sizes, and locations of the speaker, the speaker grille, the microphone, and the microphone apertureare merely examples and that other types, sizes, and locations are envisioned.

The electronic devicecan further include various sensors configured to detect proximate objects, listeners, orientations, positions, locations, and other general environment data associated with the electronic device. The position, orientation, and range of these sensors are fixed relative to a position and orientation of an active speaker of the electronic device. In particular, the electronic devicecan include a proximity sensor(e.g., capacitive, inductive, infrared, etc.) that is configured to detect the presence of nearby objects or objects in contact with the electronic device, and generate resulting proximity data. This proximity data may also reflect information regarding nearby objects relative to the speaker or speaker grille of the electronic device. The electronic devicecan further include an imaging sensorconfigured to capture optical images and generate resulting optical data (e.g., digital images). The optical data may also reflect information regarding objects relative to the speaker or speaker grille of the electronic device. It should be appreciated that various locations, types, sizes, and multiples of the proximity sensorand the imaging sensorare envisioned. Although not depicted in, it should be appreciated that the electronic devicemay further include one or more ultrasonic sensors and/or one or more infrared sensors.

Additionally, the electronic devicecan include an accelerometerconfigured to measure an acceleration, or general orientation or movement of the electronic deviceand generate resulting acceleration data, as well as a gyroscopeconfigured to measure an orientation or position of the electronic deviceand generate resulting orientation data. For example, the gyroscopemay be a three-axis gyroscope configured to measure the pitch, roll, and yaw of the electronic device. Further, the accelerometermay be a three-axis accelerometer that together with the three-axis gyroscope collectively provides six-axis capability. The electronic devicecan additionally include a location modulethat is configured to detect a location of the electronic device. For example, the location modulemay include a Global Positioning System (GPS) module. Note that Cartesian axes X-Y-Z are anchored to the electronic devicerather than the environment.

Turning now to,illustrates a back view of an example electronic device(such as a back view of the electronic devicediscussed with respect to). Again, the Cartesian axes X-Y-Z are anchored to the electronic device and thus are rotated relative to the axes indue to the fact that the electronic deviceis rotated relative to the depiction in. The electronic devicecan include an imaging sensorconfigured to capture optical images and generate resulting optical data (e.g., digital images), as well as a flash componentto assist in the capture of the optical images. The electronic devicecan further include at least one speakerwith a grilleconfigured to output audio based on an electrical audio signal. As illustrated in, the speakeris a “built-in” speaker that is commonly utilized to output audio during operation of applications such as music playback, speakerphone, and/or other similar applications (which can be the same as or in contrast to the applications used by the “earpiece” speakerof). As conventionally implemented, the speakermay be recessed within the electronic device, whereby the grilleboth protects the speakerand exposes at least a portion of the speakerto the exterior of the electronic device. Further, it should be understood that the positions and locations of the components of the electronic deviceare merely examples and that other positions and locations for the components are envisioned.

It should be appreciated that the back side of the electronic devicecan be of various shapes and sizes. For example, the back side of the electronic devicecan be flat, curved, angled, flexible, or the like. Therefore, in cases in which the back side is curved or angled, for example, the speaker grillemay not make direct contact with a surface on which (a portion of) the back side of the electronic devicerests. Of course, if the back side is flat and the back side is making direct contact with a surface, then the speaker grillemay also make direct contact with the surface unless the grille is recessed.

Althoughdepicts a single speaker, it should be appreciated that multiple speakerson the same side of the electronic deviceare envisioned, such as two speakers arranged as left-right stereo speakers, two speakers on opposite sides of the electronic device, or two speakers ported out the sides of the electronic device. Further, it should be appreciated that speakersmay be disposed or located on another surface or side of the electronic device, such as a bottom surface of the electronic device. It should further be appreciated that the various sensors and duplicate audio components of the electronic device,may be multiplicatively located on multiple sides of the electronic device,or otherwise located on a side of the electronic device,that is not depicted in. For example, the front side of the electronic devicecan include the proximity sensorand the back side of the electronic devicecan also include a proximity sensor. For further example, the speakercan instead be located on the bottom or top (y-direction) of the front side of the electronic device.

Returning to, the electronic devicefurther includes a touchscreenand a processor. The touchscreenis configured to display visual content and detect touch input from a user of the electronic device. In embodiments, a user interface of the electronic devicecan include the touchscreenvia which the user may make selections and generally facilitate functionalities of the electronic deviceas well as audio and tactile components. The processormay be a singular hardware component or may include three separate processors: an application processor to manage the applications and user interfaceof the electronic device, a sensor processor or sensor hub to manage sensor data, and an audio processor to process audio data. The processoris configured to process data associated the various audio components and sensors

(e.g.,,,,,,,,,), and facilitate the audio processing and output functionalities as discussed herein. In particular, the processorinterfaces with the audio components and sensors to detect various local positions or orientations of the electronic device,, as well as whether the electronic device,is stable. Generally, when the electronic device,is stable, it may experience little to no movement relative to a supporting surface. Based on the stability and local positions or orientations, the processorcan identify corresponding equalization settings for audio data to be output. According to embodiments, the electronic device,is configured to store the equalization settings.

Further, in some embodiments, the equalization settings may be analog equalization data implemented by two or more sets of resistor-inductor-capacitor (RLC) components (generally: an amplifier circuit), where the analog equalization data is selectable by a switch (e.g., a mechanical press switch). In these embodiments, the switch may detect contact with a surface, and may accordingly modify the path of an audio signal through a corresponding amplifier circuit, whereby the amplifier circuit corresponds to an analog equalization setting.

In embodiments, the processorcan modify an identified equalization setting based on various factors or data, such as optical data generated by the imaging sensor,, audio input data generated by the microphone, proximity data, and/or accelerometer data. The processorcan apply any modified or unmodified equalization setting to an audio signal and cause the speakerto output the audio signal such that the audio signal will be equalized according to the stability, position, and/or orientation of the electronic device,and its active speakers and speaker grilles. In some implementations, the audio equalization functionalities may be performed by analog equalization components and/or digital logic.

illustrates example local positions,or orientations of an electronic device. For purposes explaining, it may be assumed that the built-in media or “main” speaker of the electronic device (e.g., the speaker) may have its grille located on the back side of the electronic device or otherwise opposite from the touchscreen. However, in general, it should be appreciated that the built-in active speaker may be located on the front, top, bottom, or other sides of the electronic device. Further, in some embodiments, an earpiece speaker (e.g., the speaker) that is located on the front side of the electronic device may constitute the active speaker via which audio may be output, in lieu of or in addition to a back side built-in speaker. Additionally, the electronic device may include multiple built-in speakers located on various sides or combinations or sides of the electronic device.

The electronic device in the local positionis face up on a surfacesuch as a table, desk, counter, or any other surface capable of supporting the electronic device, with its speaker grille or covering face down on the surface. Similarly, the electronic device in the local positionis face down on the surfacewith its speaker grille face up. In embodiments in which the electronic device is in the face up local position, the gyroscope (e.g., the gyroscope) and the accelerometer (e.g., the accelerometer) detects that the main speakeris oriented in a downward direction with respect to gravity and stable, and the proximity sensor (e.g., the proximity sensor) does not sense proximate contact with any surface. Then, the processor (e.g., the processoror other logic circuitry) can determine that the electronic device is face up and stable and infer that it is supported relative to the surface. Accordingly, the processor can identify a “stable, face up” equalization setting corresponding to the local position. In particular, the “stable, face up” equalization setting can account for the downward orientation of the speaker (i.e., the speaker grille is oriented towards the surface). It should be appreciated that the processor may determine the “stable, face up” configuration using other combinations of sensors. As discussed herein, if the speaker grille is oriented towards the surface, the speaker grille may or may not make direct contact with the surface.

In embodiments in which the electronic device is in the local position, the gyroscope and the accelerometer detects that the speaker is oriented in an upward direction with respect to gravity and stable, and the proximity sensor (e.g., the proximity sensor) senses proximate contact with the surface. In this case, the processor can infer that the electronic device is face down and stable. Accordingly, the processor can identify a “stable, face down” equalization setting corresponding to the local position. In particular, the “stable, face down” equalization setting can account for the upward orientation of the speaker (i.e., the speaker grille is not oriented towards the surface).

In some embodiments, the electronic device can additionally or alternatively retrieve data from one or more imaging sensors (e.g., the imaging sensors,) to help determine the local position. For example, if a rear-facing imaging sensor (e.g., the imaging sensor) detects a dark local environment and a front-facing imaging sensor (e.g., the imaging sensor) detects a light local environment, the processor can determine that the electronic device is face up (and vice-versa for a face-down determination). Further, as described in more detail below, the one or more imaging sensors may be configured to analyze image data of the listener to determine the angle of the electronic device relative to the listener.

There may be situations in which there is conflicting sensor data related to the stability and/or orientation of the electronic device. For example, a proximity sensor and an ultrasonic sensor may sense conflicting surface proximity data. In these situations, the electronic device may support a hierarchy or priority that dictates which sensor data to use and which sensor data to disregard. It should be appreciated that the hierarchy or priority may order the various sensors of the electronic device in any order, and may be a default setting and/or configurable by a user.

illustrates an example local positionor orientation of an electronic device. Specifically, the electronic device in the local positionhas one edge supported by a horizontal surfaceand another edge supported by a vertical surfacesuch that neither the front side nor the back side of the electronic device makes direct contact with one of the supporting surfaces,. It should be appreciated that other orientations in which the electronic device is indirectly supported by multiple surfaces is envisioned.

In some embodiments in which the electronic device is in the local position, the gyroscope (e.g., the gyroscope) and the accelerometer (e.g., the accelerometer) detects that a speaker grilleis stable and at least partially oriented toward the supporting surfaces,, and the proximity sensor (e.g., the proximity sensoror the proximity sensor) does not sense direct or proximate contact with any of the surfaces,. Then, the processor (e.g., the processor) can determine that the electronic is face up, stable, and indirectly supported. Accordingly, the processor can identify a “stable, face up, indirect support” equalization setting corresponding to the local position. In particular, the “stable, face up, indirect support” equalization setting can account for the speaker grilleoriented toward the supporting surfaces,but not making direct or proximal contact with the supporting surfaces,. In embodiments, the proximity sensor may be located next to or proximate the speaker grilleto sense or detect when the covering of the speaker grilleis in close contact with either of the supporting surfaces,.

illustrates an example local positionor orientation of an electronic device. Specifically, the electronic device in the local positionis supported by an example mount, dock, or holder. For example, the mountmay be a vehicle windshield mount that provides support for the electronic device when traveling in a vehicle. As illustrated in, the mountsupports the electronic device on its bottom and sides and allows a speaker grilleand corresponding speaker to be exposed (but oriented away from a listener such as a vehicle driver or passenger). In some embodiments, the electronic device can sense that it is supported by the mountvia a Hall effect magnet or other type of sensor or component configured to detect when the electronic device is connected or secured to the mount(e.g., USB, NFC, ultrasound, custom connector, etc.). In other embodiments, the electronic device can determine that it is supported by the mountfrom gyroscope orientation data and/or accelerometer acceleration data. For example, if the orientation data indicates that the “x” or “y” dimension of the electronic device is “up” (i.e., the electronic device is perpendicular or near perpendicular to gravity) with a relatively small amount of “tilt,” then the electronic device can conclude that it is supported by the mount. In embodiments, the electronic device can determine different orientations, such as if the mountenables both portrait and landscape orientations.

In some embodiments in which the electronic device is in the local position, the gyroscope (e.g., the gyroscope) and the accelerometer (e.g., the accelerometer) detects that the electronic device is stable or supported, and a location module (e.g., the location module) detects that the electronic device is in motion. Then, the processor (e.g., the processor) can determine that the electronic device is stable and supported by the mount. In some cases, the acceleration data from the accelerometer can imply both that the electronic device is stable and that the electronic device is in motion (i.e., not stationary), for example in a vehicle. Accordingly, the processor can identify a “stable contact/mounted” equalization setting corresponding to the local position. In particular, the “stable contact/mounted” equalization setting can account for the speaker grilleoriented away from a user or person whereby the “x” or “y” dimension of the electronic device is “up” or close to “up” with respect to gravity. It should be appreciated that there may be different equalization settings depending on the type of the mount, to account for how the speaker grilleis oriented, whether the speaker grilleis blocked by the mount, or other considerations.

Although not illustrated in the figures, it should be appreciated that the electronic device may use other types of data from other sensors to determine its local position, orientation, or environment. For example, location data from a location module such as a GPS module can indicate that the electronic device is indoors, outdoors, in a car, or in another environment. For further example, if the electronic device is connected to a wireless local area network (WLAN), the electronic device can hypothesize that it is in an indoor environment. Additionally, if the electronic device detects that it is being charged (e.g., via a USB cord), then the electronic device can deduce that it may remain in the same location or position for a period of time. Further, the electronic device may account for an auxiliary battery pack that charges the electronic device. Moreover, the electronic device can examine optical data from the imaging sensor(s) to deduce whether the electronic device is in an indoor or outdoor environment. According to embodiments, the electronic device can store or maintain equalization settings that correspond to these and other environments or local positions.

In some embodiments, the electronic device may account for multiple speakers from which to output audio data. In cases in which the electronic device has multiple internal or built-in speakers, each of the internal speakers may have an individual equalization setting. Further, the electronic device may connect to an auxiliary speaker (e.g., via a wired or wireless connection) to output audio in addition to the one or more built-in speakers of the electronic device. Accordingly, the one or more built-in speakers may output audio data that is processed according to the determined local position of the electronic device (as well as the respective equalization setting(s)), and the electronic device may further apply a default equalization to the audio data and provide that audio data to the auxiliary speaker for output. In some cases, the electronic device may apply the same equalization setting to each built-in speaker(s) as well as any auxiliary speaker(s). In other cases, the electronic device may apply different equalization settings to respective audio data that is to be output by the multiple speakers, such as in cases in which the multiple speakers are oriented differently with respect to a user, when multiple speakers are loaded differently by an adjacent surface or lack or surface, or in cases in which each speaker corresponds to a different frequency (e.g., one built-in speaker serves as a tweeter, another built-in speaker serves as a mid-range, and an auxiliary speaker serves as a woofer). In some embodiments, any equalization(s) applied to auxiliary speaker(s) may align the output(s) of the speaker(s) to any listener(s), whereby the auxiliary speaker(s) may have imaging sensor(s) configured to detect relative locations of the listener(s) so that the electronic device can send signal(s) to the speaker(s) that are time-aligned for the listener of choice.

depicts an example representationof a userpositioned near an electronic device. The electronic devicemay be supported by a mount as illustrated in. As discussed herein, the electronic devicemay include one or more imaging sensors that are configured to generate optical data corresponding to the field of view of the imaging sensor(s). For example, the electronic device may include a front-facing imaging sensor (e.g., the imaging sensor) and a rear-facing imaging sensor (e.g., the imaging sensor) that together enable a “360 degree” field of view. The electronic devicecan analyze the optical data to detect the presence of the useras well as determine the position of the userrelative to the electronic device. For example, if optical data from a front-facing imaging sensor detects the user, the electronic devicecan determine that the useris at least positioned in front of the electronic device.

To identify an appropriate equalization setting to apply to audio data given the local orientation of the electronic deviceactive speaker grille relative to its surroundings, the electronic devicecan reconcile the determined position of the userwith location or position data corresponding to one or more speakers of the electronic device. For example, if a speaker is located on the rear side of the electronic deviceand the optical data from a front-facing camera indicates that the useris positioned in front of the electronic device, the electronic devicecan identify an equalization setting that corresponds to the speaker oriented away from the user. Similarly, for the same speaker located on the rear side of the electronic device, if the optical data from a rear-facing camera indicates that the useris positioned on the rear side of the electronic device, the electronic devicecan identify an equalization setting that corresponds to the userhaving a direct line to the speaker. The optical data may also indicate multiple listeners (including the user) in proximity to the electronic device. In this case, the electronic devicemay be configured to perform various facial recognition or machine learning techniques to identify the “primary” listener (e.g., the owner of the electronic device) and may identify/apply the equalization setting accordingly. In some embodiments, if the optical data indicates multiple listeners, the electronic device may select or revert to a preferred or default equalization setting (e.g., an equalization setting corresponding to a single user in front of a stable electronic device).

The usermay also change his or her position relative to the electronic device, which can affect tonality or the quality of the audio experienced by the user. In particular, as a useras depicted inmoves on or off an axis of the speaker, the audio experienced by the user can change.depicts an example representationof a user's movements in relation to an electronic device(e.g., in the x- and/or y-directions). The example electronic deviceincludes a front-facing imaging sensorand a front-facing speaker, whereby the useris positioned facing the front side of the electronic device.

The optical data generated by the imaging sensormay indicate a change in position of the user, for example as a result of the usermoving his or her head relative to the electronic device. In any case, the usermay change his or her position in a variety of directions (,) relative to both the imaging sensorand the speaker. As the userchanges his or her position, the usermay experience varied audio playback. For example, the audio that the userhears when directly perpendicular to the axis of the speakermay be different from the audio that the userhears when positioned at an 70° angle from the axis of the speaker. Based on the change in position indicated in the optical data, the electronic devicecan dynamically modify the equalization settings to be applied to the audio signal, and can output the equalized audio data so as to account for the new position of the user. In some embodiments, the electronic devicemay maintain a lookup table of equalization settings matched to angle (e.g., in the θ and φ dimensions or their Cartesian equivalents) to account for the position of the userrelative to the speaker output axis.

depict an example signaling diagramfacilitated by an electronic device and associated with processing audio data according to local positions or orientations of the electronic device. The electronic device can include a processor(such as the processordiscussed with respect to), a sensor hub, a speaker(such as the speakerdiscussed with respect to), a gyroscope(such as the gyroscopediscussed with respect to), an accelerometer(such as the accelerometerdiscussed with respect to), a proximity sensor(such as the proximity sensordiscussed with respect to), an imaging sensor(such as the imaging sensordiscussed with respect to), and a microphone(such as the microphonediscussed with respect to). In another embodiment, simple digital logic may be used to implement the functionalities of the signaling diagram.

As illustrated in, the gyroscopemay periodically, intermittently, or continuously transmit () orientation data to the sensor huband the accelerometermay periodically, intermittently, or continuously transmit () acceleration data to the sensor hub. In embodiments, the sensor hubmay request the gyroscopeand the accelerometerfor orientation data and acceleration data, respectively. The processormay periodically, intermittently, or continuously request () a sensor-derived state of the electronic device from the sensor hub, where the sensor-derived state may be identified from the most recent orientation data and acceleration data.

The functionalities may continue with a userof the electronic device requesting () audio playback. For example, the usercan interface with a music streaming application and request to initiate playback of a playlist or song. The processorcan examine the sensor-derived state of the electronic device retrieved into determine () whether the electronic device is stable. In some cases, the processorcan deduce that the electronic device is stable if the orientation data and the acceleration data indicate that the electronic device is not moving. In other cases, the processorcan deduce that the electronic device is stable if the orientation data and the acceleration data indicate that the electronic device is in motion but is supported (e.g., if the electronic device is secured in a mount in a vehicle).

If the processordeterminesthat the electronic device is not stable (“NO”), for example if the useris holding the electronic device, the processorcan apply () a default equalization setting (or, in some cases, a handheld equalization setting) to an audio signal and provide () the audio signal with the applied default equalization setting to the speaker. The speakercan output () the audio signal with the applied default equalization setting. If the processordetermines that the electronic device is stable (“YES”), the processorcan retrieve () proximity data from the proximity sensor. Based on the proximity data, the processorcan determine () whether a portion of the electronic device (or more particularly, the proximity sensor) senses proximity to an external object or surface. If the processordeterminesthat the proximity sensorsenses proximity to the surface (“YES,” for example the local positionillustrated in), the processorcan identify () a “stable direct contact” equalization setting. In contrast, if the processordetermines that the proximity sensordoes not sense proximity to the surface (“NO,” for example the local positionsandillustrated in), the processorcan identify () a “stable indirect contact” equalization setting. In an optional embodiment, the proximity sensor(or in some cases an ultrasonic sensor or another sensor) may detect a distance from a speaker grille (such as the speaker grille) of the electronic device to the surface. In this implementation, the processormay identify an equalization setting that is additionally based on the distance from the electronic device to the surface (which may be the same as or different from the “stable indirect contact” equalization setting identified in).

In an optional embodiment, the processorcan retrieve () audio input data from the microphone. In some embodiments, the audio input data can include microphone feedback present in various environments (e.g., indoor, outdoor, etc.). The processorcan optionally modify () the equalization setting based on the audio input data. In another optional embodiment, the processorcan retrieve () optical data from the imaging sensor(and optionally from an additional imaging sensor). The optical data can detect a presence of the userand can indicate the location of the userrelative to the location of the speaker(e.g., if the useris facing the front side of the electronic device and the speakeris on the opposite side of the electronic device). Further, the optical data can indicate a change in the location of the userrelative to the axis (i.e., a perpendicular direction) of the speaker. The change in the location of the usermay also be characterized by angle(s) of the userrelative to axis(es) of the electronic device speaker, whereby the electronic device may maintain/access a lookup table storing various equalization settings based on the angle(s). According to embodiments, the usermay experience different audio tonalities as the usermoves “off” or “on” the axis of the speaker. The processorcan optionally modify () the equalization setting based on the optical data.

The processorcan apply () the modified or unmodified equalization setting to the audio signal. Further, the processorcan provide () the audio signal with the applied equalization setting to the speakerand the speakercan output () the audio signal with the applied equalization setting. In embodiments, the local position of the electronic device may change, and the processorcan dynamically identify other various equalization settings based on the change in local position. Further, the processorcan perform the equalization setting modification, audio data application, and audio output functionalities based on the changed local position.

is a flowchart of a methodfor an electronic device (such as the electronic device) to process various sensor data and facilitate audio equalization techniques based on the sensor data. The order of the steps of the depicted flowchart ofcan differ from the version shown, and certain steps can be eliminated, and/or certain other ones can be added, depending upon the implementation. The methodbegins with the electronic device receivinga request to output audio via a speaker. The request may be received from a user via a user selection or may be automatically detected via another type of trigger (e.g., a voice command, an NFC detection, a scheduled trigger, etc.).

The electronic device determinesif it is stable by examining acceleration data from an accelerometer and/or orientation data from a gyroscope. In another embodiment, the electronic device may retrieve a sensor-derived state of the electronic device from a sensor hub. If the electronic device is not stable (“NO”), the electronic device identifies a default equalization setting and appliesthe default equalization setting to an audio signal. The default equalization setting may be a standard or universal equalization setting that is used for example when the local position of the electronic device is indeterminate. The electronic device also outputs, via the speaker, the audio signal according to the default equalization setting. Processing can then return toor proceed to other functionality.

If the electronic device is stable (“YES”), the electronic device determinesif its speaker grille is contacting or proximate to a surface by examining proximity data from a proximity sensor (or via other sensors such as an ultrasonic sensor, an infrared sensor, or others). If the proximity data does not indicate contact or proximity (“NO”; e.g., if the electronic device is leaning against a surface and the speaker grille is not supported by the surface), the electronic device identifiesa stable indirect contact equalization setting. In some cases, the speaker grille may be orientated away from a support surface, whereby the electronic device may identify an away-oriented equalization setting. If the proximity data does indicate contact or proximity (“YES”; e.g., if the electronic device is laying flat on a surface and the speaker grille is facing the surface), the electronic device identifiesa stable contact equalization setting. In embodiments, the electronic device can also identify the equalization setting based on the orientation of the speaker (e.g., if the speaker (and corresponding speaker grille) is face up or face down). In some cases, the electronic device may not be able to determine whether it is stable, in which case the electronic device may estimate its distance to the surface (e.g., via proximity data) and apply equalization data accordingly.

In an optional embodiment, the electronic device retrieves audio input data, for example from a microphone, and modifiesthe identified equalization setting based on the audio input data. For example, if the microphone detects background noise, the electronic device can modify the equalization setting to account for the background noise. The electronic device further determinesif there is any optical data, such as if an imaging sensor detects a presence of a user positioned relative to the speaker. If optical data is not available or no user is found in the optical data (“NO”), the electronic device appliesthe identified equalization setting to the audio signal. If there is optical data and a user is found in the optical data (“YES”), the electronic device modifiesthe identified equalization setting based on the optical data. For example, the optical data can indicate that the user is positioned on a side of the electronic device opposite from the speaker. For further example, the optical data can indicate that the user is positioned at a certain angle relative to an axis of the speaker. The electronic device appliesthe modified equalization setting to the audio signal.

After applying the equalization setting, the electronic device outputs, via the speaker, the audio signal according to the applied equalization setting. The electronic device also determinesif there is a change in the optical or stability data, such as if the user adjusts his or her position relative to the speaker. In this case, the imaging sensor generates updated optical data corresponding to the user's change in position. If there is only a change in the optical data(“YES”), processing may return toin which the electronic device modifies the equalization setting based on the updated optical data. If there is a change in data other than optical data (such as a change in stability data) (“NO”), processing may return to. The determinations ofandmay be may be repeated periodically or when triggered by a change in optical or stability inputs. If there is not a change in the optical or stability data in(“NO”), processing may end or proceed to other functionality. In some embodiments, the electronic device may further modify the equalization setting based on GPS location and/or acoustic loading.

illustrates an example electronic device(such as the electronic devicediscussed with respect to, or other devices) in which the functionalities as discussed may be implemented. The electronic devicecan include a processoror other similar type of controller module or microcontroller, as well as a memory. The processormay include a singular processor or may include more than one separate processor such as: an application processor to manage the applicationsand user interfaceof the electronic device, a sensor processor to manage sensordata, and an audio processor to process audiodata. The memorycan store an operating systemcapable of facilitating the functionalities discussed. The processorcan interface with the memoryto execute the operating systemand a set of applications. The set of applications(which the memorycan also store) can include an audio equalization applicationconfigured to process audio data according to the techniques discussed. The set of applicationscan also include one or more other applicationssuch as, for example, music and entertainment applications, phone applications, messaging applications, calendar applications, social networking applications, utilities, productivity applications, games, travel applications, communication application, shopping applications, finance applications, sports applications, photography applications, mapping applications, weather applications, applications for connecting to an online marketplace, and/or other applications.

The memorycan further store a set of equalization settingsthat correspond to various local positions or orientations of the electronic device. According to embodiments, the audio equalization applicationcan interface with the equalization settingsto retrieve appropriate equalization settings to apply to audio data. Generally, the memorycan include one or more forms of volatile and/or non-volatile, fixed and/or removable memory, such as read-only memory (ROM), electronic programmable read-only memory (EPROM), random access memory (RAM), erasable electronic programmable read-only memory (EEPROM), and/or other hard drives, flash memory, MicroSD cards, and others.

The electronic devicecan further include a communication moduleconfigured to interface with one or more external portsto communicate data via one or more wired or wireless networks. For example, the communication modulecan leverage the external portsto establish a wide area network for connecting the electronic deviceto other components such as a remote data server. According to some embodiments, the communication modulecan include one or more transceivers functioning in accordance with IEEE standards, 3GPP standards, or other standards, and configured to receive and transmit data via the one or more external ports. More particularly, the communication modulecan include one or more WWAN, WLAN, and/or WPAN transceivers configured to connect the electronic deviceto wide area networks (e.g., to receive steaming music that may be pre-equalized for the electronic device), local area networks, and/or personal area networks. The electronic devicemay further use one of the external portsto connect to peripheral or auxiliary components such as an auxiliary speaker.

The electronic devicecan further include one or more sensorssuch as one or more accelerometers, gyroscopes, imaging sensors, proximity sensors, a presence sensor (not shown in), one or more ultrasonic sensors (not shown in) and location modules. The sensorsmay also include other types of sensors such as light sensors, infrared sensors, touch sensors, NFC components, and other sensors.