Location based retail media and ad attribution system

This application claims the benefit of U.S. Provisional Application No. 63/507,244, filed Jun. 9, 2023, and U.S. Provisional Application No. 63/512,751 filed Jul. 10, 2023, and U.S. Provisional Application No. 63/570,487 filed Mar. 27, 2024, and U.S. Provisional Application No. 63/512,754 filed Jul. 10, 2023, and U.S. Provisional Application No. 63/520,159 filed Aug. 17, 2023, and U.S. Provisional Application No. 63/570,500 filed Mar. 27, 2024, and U.S. Provisional Application No. 63/515,928 filed Jul. 27, 2023, and U.S. Provisional Application No. 63/515,932 filed Jul. 27, 2023, and U.S. Provisional Application No. 63/515,936 filed Jul. 27, 2023, and U.S. Provisional Application No. 63/570,492, filed Mar. 27, 2024, and U.S. Provisional Application No. 63/515,939 filed Jul. 27, 2023, and U.S. Provisional Application No. 63/520,160 filed Aug. 17, 2023, and U.S. Provisional Application No. 63/515,942 filed Jul. 27, 2023. The entire disclosures of the above applications are incorporated herein by reference.

The present disclosure relates generally to a retail monitoring and tracking system and, more specifically, to various methods and systems for providing an improved user experience and allowing retailer to have the benefit of further data.

This section provides background information related to the present disclosure which is not necessarily prior art.

A retail experience for a customer has the customer moving through the retail environment and placing products within a cart. Various fixed displays are provided by the retailer or the consumer packaged goods (CPG) provider. Besides purchase data, very little data is available to the merchant relative to the products, the consumer purchasing the product and various other types of data. Consumers, on the other hand, beside the store displays that are fixed are provided with no customized information regarding purchases.

This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.

The present disclosure provides a method and system for providing useful information to consumers in a retail environment to encourage purchases and request further information regarding purchases. Further, the retailer is also provided various types of data that allows the retailer to provide targeted marketing, store layout and other improvement to ultimately help the consumers.

In one aspect of the disclosure, a method and system for providing advertisements to user device includes determining a first location of a user device within a premises. The user device is associated with a user device identifier. The method further includes communicating the user device identifier to an on-premises controller, matching a user profile to the user device identifier, identifying a product associated with the first location of the user device, communicating product data associated with the product to a controller, and communicating an advertisement to the user device from the controller based on the product data.

In another aspect of the disclosure, a system includes an on-premises controller determining a first location of a user device within a premises. The user device is associated with a user device identifier. A user device communicates the user device identifier to an on-premises controller. The on-premises controller matches a user profile to the user device identifier, identifies a product associated with the first location of the user device, receives product data associated with the product, updates the user profile to form an updated user profile based on the product data, and communicates an advertisement to the user device based on the updated user profile.

Further areas of applicability will become apparent from the description provided herein. The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.

Example embodiments will now be described more fully with reference to the accompanying drawings.

10 12 12 14 16 12 18 16 20 20 22 22 20 20 24 24 24 26 24 12 20 24 1 FIG. A retail control systemis illustrated having a premises. A premisesmay be defined by wallsof a building that has an indoor portion. The premisesmay also include other portions of a retail location including an outdoor locationand portions of the parking lot, outbuildings or sidewalk outside. A very simplified view of an indoor portionis set forth in. The indoor portion may include a plurality of shelving units. The shelving unitsmay be positioned to leave aislestherebetween. The aislesallow areas for consumers to pass so that products may be obtained from the shelving units. The shelving unitsmay also have display casesor the display casesmay be separate items. The display casesmay have display case doorsthat can be locked to protect valuable products from theft. Various numbers of display casesmay be used throughout a premises. All of the shelving unitsmay be replaced by display casesin high crime areas.

30 12 30 32 32 30 30 36 30 A plurality of ranging sensorsmay be provided around the premisesfor determining the user or user device location. The ranging sensorsmay receive signals from a user device. The user devicemay communicate signals that include a user device identifier. The technology may be one technology for all of the ranging sensorsor a variety of types of technologies. For example, a Wi-Fi sensor, an RF sensor, a RSSI sensor, a sounding sensor, a Bluetooth sensor, a BLE sensor or a UWB sensor are examples of suitable types of ranging sensors. Ultimately, the ranging sensorsprovide ranging signals to an on-premises controller. The ranging sensorsmay provide a location signal and a direction of travel signal. For some technologies such as Bluetooth® the identifiers may change over time.

38 12 16 18 38 38 12 38 30 32 38 32 40 Camerasmay also be located within the premisesin the indoor portionand the outdoor portion. The camerasmay be mounted to the walls or ceiling and positioned to provide coverage throughout the premises. Overlapping fields of viewA and near complete coverage in the premisesmay be provided depending on the spacing of the cameras. The ranging sensorsmay provide location information based on the position of the user device. The camerasprovide more detailed information, such as orientation and allowing the user deviceto be correlated with the position of a user.

41 38 30 41 The store may be divided into various zones, each zone may not necessarily correspond to a single field of viewA or one ranging sensor. The Zonesmay correspond to a zone classification associated with one or more products on the shelf. As described later, a planogram may be used in the determination of which products are in which zones.

38 30 42 42 42 As will be described in greater detail below, the combination of the fields of viewA and the output of the ranging sensorsmay provide the detailed location and therefore the user device journey data for a journey. The journeymay also have a time component associated therewith as is also described below. A dwell time at certain locations allows the products associated therewith to be associated with the journey.

42 40 50 52 40 52 24 At the end of the journey, certain examples in the detail description set for the below have the usercheck out at a point of sale device. The point of sale device may have a scannerfor scanning the content being purchased by the user. Scanning by the scannermay verify the contents removed from a display case.

58 59 40 40 40 30 38 A displayand speakermay be used to display and communicate an audible signal or the advertisement to the user. The advertisements may be directed to the userbased upon a user profile of the userand the location determined by the ranging sensorsand the cameras.

30 38 36 60 62 The ranging sensorsand the camerasmay be wired and connected directly to the on-premises controller. However, wireless communications through a routermay form the network.

2 FIG. 10 36 30 38 58 59 32 24 62 62 60 32 36 60 32 32 32 32 32 10 70 70 Referring now to, the retail control systemis illustrated in block diagrammatic form. The on-premises controlleris illustrated coupled to the ranging sensors, the cameras, the display, the speakerthe user device and an employee user deviceA and a display casethrough the network. As mentioned above, the networkmay be wired or wireless. One routeris illustrated between a user deviceand the on-premises controller. However, all of the interconnections may be made through one or more routers. As will be described in greater detail below, the employee user deviceA may be configured in a similar manner to the user devicewith the intent to be used by a store employee. As will be described below, the employee user deviceA may receive directions to locate the user deviceto allow the employee of the store to offer assistance to the user of the user device. The systemmay also include a shopping cart or basket device. The shopping cart or basket devicemay include electronics for locating as described in greater detail below.

10 80 80 36 62 80 36 80 The systemmay also include an off-premises controller. The off-premises controlleris in communication with the on-premises controllerthrough the network. The off-premises controllermay be in communication with one or more on-premises controllers. The off-premises controllermay control, for example, various high-level functions such as controlling the user profile for various users, providing advertisements to users once they are off-premises and several other functions as described in greater detail below.

80 84 84 80 84 80 The off-premises controllermay also be in communication with a consumer packaged goods (CPG) controller, which is a supplier or manufacturer of the products sold in the store. The CPG controllermay receive data from the off-premises controllerto allow improvements to be made to the goods or products provided by the CPG provider. Packaging, advertisements, placement and other types of modifications may be made by the CPG controllerbased upon the data received from the off-premises controller.

3 FIG. 32 310 312 32 314 312 314 316 316 317 314 318 312 Referring now to, the user devicemay be a mobile device that has an applicationthat is controlled by a user device controllerto perform various functions. The user devicehas a user interfacecoupled to the controllerfor inputting data. The user interfacemay be physical buttons, a keyboard, switches or a touch screen device that may be part of a display. The displaymay have a speakerassociated therewith. Audible, visual or both audible and visual advertisements may be provided. That is, the user interfacemay have virtual buttons, dials or slides that are used for controlling various functions including inputting data to form a user profile in accordance with the profile controllerof the controller.

32 320 312 320 322 312 312 322 322 312 322 324 324 324 The user devicemay also include a biometric scannerthat is used for providing biometric data to the controller. The biometric datamay include but is not limited to facial recognition data or fingerprint data or combinations thereof. A cameramay be used to provide biometric data such as facial data to the controller. The controllermay use the camera data from the camerato determine a location, orientation or even nearby products, for example. The cameramay also be used to input biometric data such as facial data that is communicated to the controller. The cameramay also have a scan tool. The scan tooluses images to request data or provide data relative to the product being contemplated for purchase or purchase. The scan toolmay scan the items being placed into a cart so that scanless checkout may be performed.

326 312 326 A microphonemay be used to provide data to the controller. The microphonemay be used to input voice commands for requesting different functions or providing data for different functions.

328 62 330 62 328 328 62 328 A transceivercommunicates and receives data signals to and from the network. A network interfacemay be used to provide the connection to the networkto and from the transceiver. The transceiveris both a transmitter to communicate data to the networkand has a receiver that is used to receive data from the network. Both a transmitter and receiver may be separately provided. The transceivermay also generate signals such as Wi-Fi signal, RF signals, RSSI signals, sounding signals, Bluetooth signals, BLE signals or UWB signals that are ultimately communicated to a ranging sensor to allow a user device identifier to be sent and a range or location to be determined based on the signals.

332 30 332 30 12 A range devicemay be used for communicating data signals to the ranging sensor. The range devicemay thus generate a signal that includes a user device identifier that is communicated to the ranging sensorsso that the user device may be identified, and triangulation techniques may be used to determine the location of the user device within the premises.

32 340 340 32 12 340 32 20 24 340 340 340 32 The user devicealso has a plurality of device sensors. The device sensorsare used for various purposes including determining the location of the user devicewithin the premises. The device sensorsmay fine tune the location and/or orientation of the user devicerelative to the shelving unitsor the display casesor both. The device sensorsmay include, but are not limited to, an accelerometerA that generates acceleration signals in various directions. Typically, accelerators may be provided in three axial directions such as in a X direction, a Y direction and a Z direction. The accelerometerA may help locate the user deviceas well as providing movement signals.

340 32 340 32 An ambient light sensorB generates a signal corresponding to the amount of light at the user device. The ambient light sensorB may help determine if the user deviceis inside or outside of pocket/purse/etc.

340 A magnetometerC is used to generate signals corresponding to the orientation of the user device within a magnetic field, such as the magnetic field of the earth.

340 340 32 A global positioning sensorD (GPS sensorD) generates global positioning signals corresponding to the position of the user devicerelative to the surface of the earth as determined by triangulation from signals emitting from satellites.

340 32 340 A gyroscopeE generates gyroscope signals corresponding to the relative position of the user device. The gyroscope signals may include acceleration signals in various directions as well as moment signals about the various axes. The gyroscopeE may therefore have X, Y and Z axes and generate acceleration signals corresponding thereto and the moment signals about each of those axes.

340 340 340 The device sensorsmay also include an altimeterF. The altimeter signals from the altimeterF correspond to the altitude of the user device relative to sea level.

340 340 The device sensorsmay also include a proximity sensorG that generates a signal corresponding to the proximity to a nearby object. The proximity signal may provide a distance or a relative measure to an object.

312 318 318 The controllermay be used for various functions including receiving user profile information such as various user information, user attributes and the like. The profile controllermay, for example, provide basic information such as establishing passwords, user residence location and the like. In addition, the profile controllermay store a plurality of user attributes such as the skin type, the hair type, other physical or medical attributes that allow marketing for specific products.

312 350 352 354 352 310 352 The controllerhas a microprocessoror “processor” that is in communication with a memoryand a timer. The memorystores the programming, data and the like for performing various functions including for operating the application. The memorymay be a non-transitory computer-readable medium that includes machine readable instructions that are executable by the processor. The machine readable instructions include allowing the performance of various functions.

312 356 356 352 356 356 32 356 12 356 24 12 24 12 24 356 The controllermay also have an authentication level memory. The status level memoryis illustrated and is a separate memory from memoryalthough the memorymay be incorporated therein. The authentication level memorymay set an incentive level associated with the user device. By way of example, the authentication level memorymay provide levels of incentives to enable various functions within a premises. For example, the authentication level memorymay provide a first level that allows the opening of all of the display caseswithin a premises. A medium or mid-level status may provide access to less than all of the display caseswithin a premises. A third authentication level may allow access to even fewer display casesthan the mid-level status. The authentication level memorymay be used to store the authentication level that provides discounts or the like. Authentication levels may be achieved by various security clearances or the amount of spending at a retail chain.

360 12 360 360 360 360 360 32 An attribution controllermay be used to determine the attributions of a visit to a premises. The attribution controllermay determine a dwell timeA or provide face tracking atB. By providing dwell timeA and face trackingB, the locations of the user device and the orientation of the user devicemay be determined.

364 24 356 364 356 A key generatormay generate and/or store a key for accessing various functions including accessing the display. As mentioned above, the authentication level memorymay also limit the operation of the key generatorto allow the key to order when a certain authentication level, status or incentive level is achieved at the authentication level memory.

366 362 A payment controlleris used to control the payment to the point of sale device or without the point of sale device. That is, a credit card or bank account information may be stored at the payment controller and communicated through a user interface or a range deviceto make payment for various products.

368 368 368 352 An age memorymay store the age level of the user associated with or assigned to the user profile. The age level stored in the memorymay be used for marketing and for access purposes. For example, cigarettes and alcohol may be regulated by governmental authorities and therefore access to certain display cases below a certain age may be restricted. The age memorymay be incorporated into the memory.

4 FIG. 1 FIG. 36 36 410 412 414 412 410 412 36 36 420 421 62 36 422 422 30 12 422 12 422 32 32 422 42 12 422 32 38 32 424 32 12 38 38 12 38 422 424 12 424 Referring now to, the on-premises controlleris illustrated in further detail. The on-premises controllerincludes a microprocessor, a memoryand a timer. The memoryis a non-transitory computer-readable medium including machine-readable instructions that are executable by the processor. The machine-readable instructions stored within the memoryallow various functions to be performed by the on-premises controllerVarious functions are described herein and later in conjunction with the flowcharts. The on-premises controllerhas a network interfaceand transceiverthat are used to communicate to and from various devices through the networkillustrated above. The on-premises controlleralso includes a ranging sensor controller. The ranging sensor controllerreceives ranging signals from the ranging sensorsdistributed throughout a premises. The ranging sensor controllerreceives ranging signals that originate from the various user devices within the premises. The ranging sensor controllermay receive several signals from different ranging controllers that have a user identifier corresponding to the user identifier of a particular user device. Based upon the signal strengths, the angle of arrival, the time difference between the times of arrival of the radio signal and the like, a location and a direction of travel of the user devicewithin a premises may be determined. The ranging sensor controllermay provide various locations ultimately to keep track of the journeywithin the premises. The ranging sensor controllermay or may not be able to make extremely precise determinations as to the location of the user device. The plurality of camerasmay be used to increase the accuracy or supplement the location of the user devices. Therefore, a camera controllermay also be used to determine locations of the user devicewithin the premises. As is illustrated in, the camerashave a certain field of viewA that may or may not cover every area or zone of the premisescompletely. The field of viewA may be blocked or partially blocked in that different perspective angles may prevent a complete view from occurring. Therefore, the ranging sensor controllerand the camera controllerand the locations determined thereby are complementary to form a more complete determination of the location of all the user devices within the premises. The camera controllergenerates camera image signals that are electrical signals that correspond to camera images generated by the camera or video generated by the camera. The camera signals may be still images or a series thereof or video images both of which are analyzed for the objects therein.

422 424 426 426 426 428 422 424 430 The ranging sensor controllerand the camera controller, and the signals generated thereby are used by the objector identifier controllerto identify objects therein. The object identifier controllermay be used to generate objects corresponding to the users and/or user devices. The object identifier controlleridentifies the objects within an image. A dwell time controllerdetermines the dwell time of the devices from the ranging sensor controllerand the camera controller. Ultimately, a location controllerdetermines the location of the user device along with a dwell time at each of the locations.

432 42 36 438 438 438 440 440 58 59 440 430 1 FIG. A journey controllerdetermines a journey with the locations and the dwell time of the user and/or user device within the premises. The locations and dwell times form the path or journeywhich is a time-based sequence. The journey data may be based on at least one of a plurality of locations within the premises, the plurality of locations within the premises and a dwell time at each of the plurality of locations, a walking speed, heading and a comparison of dwell events to purchases The journey may include user-product interactions and various service points within a store such as a pharmacy, optical, and various vending/service machines. An on-premises controllermay receive a profile of the user from the profile controller. The profile controllermay match the user identifier and profile stored within the profile controller. The profile controllerallows the advertisement controllerto provide targeted advertisements based on user characteristics. The advertisement controllermay control the advertisement on the displayand or speakerillustrated in. Suggestions may be provided by the advertisement controllerbased upon various aspects of the location determined by the location controller.

442 444 446 440 448 448 450 450 452 440 58 440 316 32 80 3 FIG. A planogram controllerprovides a product placement identifier in conjunction with a map. Products may be associated with various locations by the product identifier. Based on the dwell times, products may be recommended by the advertisement controller. A comparison controllercompares various locations and the products therein. The comparison controllermay act in conjunction with a purchase controller. For example, based upon the locations visited and the dwell times determined by the dwell time controller and purchases the user purchases as determined by the purchase controller, recommendations may be made on-premises for various advertisements to be presented to the user. The product information databasemay provide specific product information to be advertised to the user. The advertisement controllermay not only control the advertisements provided at the display. The advertisement controllermay also provide data to the displayof the user deviceas illustrated in. The data relative to the dwell time, the locations and the journey of the user device may be communicated to the off-premises controlleras described in greater detail below.

36 452 454 442 444 454 444 456 32 458 32 458 456 444 458 458 The on-premises controllermay also be used to provide not only product information from the product information database, but inventory amounts from an inventory database. The planogram in the planogram controllerand the maptogether with the inventory databaseallows the user to search for a specific product, obtain product information, locate the product on the mapand allow a routing controllerto route a user or user devicetoward the desired product. Should assistance be required by the user, the employee selector controllermay be used to allow a retail establishment employee to reach the position of the user device. The employee selector controllermay use the routing controllerand the mapto provide instructions to reach the user device in need of assistance. The employee selector controllermay also have details as to the expertise of various employees. For example, in a home improvement environment, the employee selector controllermay route an employee with knowledge of plumbing to a user device in the plumbing department. In an employee device, the profile may be established for various levels of expertise for various employees. Of course, other functions may be enabled by the on-premises controller including the details provided in the flowcharts below.

464 24 32 24 464 464 466 466 32 46 466 36 24 34 464 464 A key generatormay be used to generate a key used to access the display case. That is, when a user devicerequires access to the display case, a key may be provided by the key generatorto the user device. The key may be a digital code. The key generatormay also provide a key to a display case controller. The display case controllerreceives the corresponding key so that when the user deviceis brought within a predetermined range of the display case, the display case controlleruses the key to unlock the display case. The display case controllermay control the locking and unlocking of the display case from the on-premises controllerrather than locally at each display case. That is, a control signal may be provided from the display case controller to a locking actuator of the display caseto allow access to the display casefor the user that presents the key from the key generator. The key generatormay provide an expiration time for the key. The key may expire or be revoked after use or within a predetermined time period after use.

5 FIG. 80 80 510 512 514 512 Referring now to, the off-premises controlleris illustrated in further detail. The off-premises controlleralso includes a microprocessor, a memoryand a timer. The memoryis a non-transitory computer-readable medium that includes machine readable instructions that are executable by the processor to perform various functions described herein.

80 520 521 62 62 520 80 The off-premises controllerincludes a network interfaceand a transceiverthat are used for interfacing with the network. As mentioned above, the networkmay be a wired or wireless network, the network interfacemay be used for interfacing with one or more different premises or user devices once they leave a premises. For example, one off-premises controllermay be used to communicate with a number of different stores throughout a chain of stores.

522 522 430 428 432 522 A journey controllermay be used to track the user devices within a premises. The journey controllermay provide the tracking based on the locations from the location controller, the dwell time controllerand the journey controller. The journey controllermay also be simplified in tracking the journeys from the various devices within various premises.

524 80 36 32 524 62 36 32 62 A product databasemay also be located on the off-premises controllerrather than the on premises controller. When product information or data is requested from the user device, the product databasethrough the networkmay provide the data to the on-premises controlleror directly to the user devicethrough the network.

526 80 526 12 526 36 526 A profile controllermay also be included within the off-premises controller. For a retail chain with many stores, the profile controllermay store the profiles of the various users in the system including the data associated therewith to allow the users to visit any store and have the same experience. When a user is identified within the premises, the profile controllermay be contacted by the on-premises controllerto receive the information for the associated user. A profile controllermay also be used to provide various correlations to various user devices, groups of user devices or target marketing groups of user devices having various characteristics. For example, older users may be provided information regarding products directly to older customers.

528 12 530 528 522 530 A purchase data controllermay aggregate purchases for each premisesThe aggregate of the purchase data allows various trends in consumer tastes to be determined, correlations to be made between product placement and the popularity of various products. The planogram controlcan correlate the purchase data from the purchase data controller, the location dwell time and journey of various user devices as determined or received at the journey controllerto allow planogram changes to be recommended from the planogram controller.

532 534 80 528 534 An advertisement effectiveness controllertogether with an advertisement controllermay be used to control various advertisements that are communicated to a user device. As will be described in greater detail below, advertisements may be provided during the shopping experience. However, once the user device leaves the premises, further ads may be communicated to the user device from the off-premises controllerbased on different data regarding the user. The ads may allow retargeting advertisements based upon various characteristics. For example, when a user dwells at a particular area without buying a product (as determined by the purchase data controller), the advertisement controllermay generate advertisements corresponding to the unpurchased products. However, when a product is purchased after a long dwell time at a location, advertisements may not be provided to the user device after leaving the premises.

532 528 532 532 532 534 The advertisement effectiveness controllermay receive data from the purchase data controlleras to the effectiveness of various advertisements within or outside the premises. The advertisement effectiveness controllermay assess the effectiveness of advertisements provided in the store by determining whether advertised products have been purchased after a targeted advertisement. The advertisement effectiveness controllermay also be used off-premises. When advertisements are provided to a user off-premises, when a product is purchased that was advertised off-premises, the next time the user enters the store, the advertisement will be deemed effective. The advertisement effectiveness controllerand the advertisement controllermay be programmed to prevent repeating advertisements in a repetitive manner.

540 540 32 A payment controllermay be used to control the payments for a user. Together with the user profile, the payment controllermay provide credit card or bank account information to the on-premises controller when the user deviceand the user associated therewith is purchasing products at the point-of-sale device or otherwise.

6 FIG. 70 32 610 70 30 70 70 612 613 612 613 Referring now to, the shopping cart or basket within the store may also be a smart device which may become a type of user device because it may be associated with a user during a shopping experience. Electronics may be coupled thereto to provide identification of a user. The features of the shopping cart or basketmay be less than that of a user devicein that personal details may not be provided. However, an identifier controllermay be used to communicate an identifier corresponding to the shopping cart or basketto the ranging sensors. Therefore, the shopping cart or basketmay be tracked. The shopping cart or basketmay also have a displayand a speakerassociated therewith. The displayand or the speakermay provide or display purchase data, product information and the like.

614 62 36 616 612 616 612 612 618 618 616 618 70 614 614 620 A network interfaceinterfaces with the networkof the premises to allow the advertisements and intercommunication with the on-premises controller. A user interfacemay be provided to allow a user to input various data including making selections and moving through various advertisements on the display. The user interfacemay be a plurality of buttons or virtual buttons on the displaywhen the displayis a touch screen display. A biometric interfaceallows biometric data to be provided therethrough. The biometric interfacemay allow fingerprinting or facial recognition to be used to identify a user. Ultimately, the user interfaceand the biometric interfaceallow a user to identify with the shopping cart or basketwhich may act as a user device described above in identifying the user. The network interfacemay allow a user profile to be used from a profile controller to allow target ads and other information to be used by the shopping cart or basket. The network interfacethrough the transceiverallows two-way communication with the network.

622 36 624 70 624 616 A position detectorcommunicates with the ranging device to provide position data to the ranging devices and ultimately to the on-premises controller. A camera/scannerallows various products to be scanned when placed within the shopping cart or basket. Therefore, the camera/scannermay be used to inventory the cart and allow a user to pay without a cashier. After paying, the user interfacemay be used to remove any identifying information from the shopping cart.

70 630 632 634 630 630 632 632 The shopping cart or basketincludes a controller microprocessor, a memoryand a timer. The microprocessormay be referred to as a processor. The microprocessortogether with the memorycontrols various functions. That is, the memoryis a non-transitory computer readable medium that includes machine-readable instructions that are executable by the processor to perform the various functions.

7 FIG. 24 24 710 712 714 716 712 712 714 710 720 720 30 720 24 720 722 730 732 732 730 730 732 36 32 732 732 Referring now to, further details of the display caseare set forth. The display caseincludes a controllerthat has a microprocessor, a memoryand a timer. The microprocessormay be referred to as a processor. The microprocessortogether with the memorycontrols various functions. That is, the memory is a non-transitory computer readable medium that includes machine-readable instructions that are executable by the processor to perform the various functions. The various functions relative to the product display case allow inventory control, access control and the like. The controllermay be coupled to a ranging sensor. The ranging sensormay act as a ranging sensordescribed above. However, the ranging sensormay be incorporated into the display case. The ranging sensorallows the proximity to be determined by the proximity determination controller. When the user device is very close, as is determined by design, a lock controllermay be used to control a lock actuator. The lock actuatormay be latched and unlatched by control of the lock controller. The lock controllermay unlatch the latch actuatorwhen the key generated by the on-premises controlleror the user devicematch. The lock may be locked by the lock actuatorand the controller after a time period timed by the timer times out. Likewise, the lock actuatormay be controlled after the lock is unlocked or locked if the door is not opened after a time period.

740 710 744 746 744 746 24 744 748 750 748 62 748 36 80 24 752 753 752 753 752 753 752 753 752 A door status sensorprovides a door status symbol to the controller. The door status sensor may provide an open signal, a closed signal or signals that correspond to a transition between the open and closed state or a closed and open state. The door status sensor may be used together with an inventory controllerand a product identifierto determine the inventory within the product display case. That is, the inventory controllermay generate or count the products within the display case prior to opening the display case. The product identifiermay be an RF identifier that energizes and receives signals from tags on the products within the display case. Prior to opening a display case may determine the inventory. After the door opening, a second inventory may be determined and the inventory controllermay communicate the change in inventory through an RF transceiverand a network interface. In this manner, the user account may be charged. The transceivermay be used to communicate with user devices directly through short range communications such as Bluetooth or indirectly through the network. The transceiveralso communicates with the on-premises controlleror the off-premises controller. The product display casemay also include a displayand a speaker. The displayand speakermay be used to communicate instructions and/or communicate advertisement data to a user. For example, a display caseand/or a speakermay instruct a user to communicate a key or move the user device closer so that a key is communicated. The displayand/or the speakermay generate instructions for opening the case or that the key has been received successfully. The displaymay also generate warnings including that the customer is responsible for all items removed and prevent unauthorized/unverified shoppers nearby from accessing the product display case.

754 754 754 A scannermay also be incorporated into the product display case. The scannermay be a bar code scanner or an RF ID scanner. The scannermay be actuated by the opening of a display case door or switch electrically couple thereto.

8 FIG. 810 812 812 Referring now to, a method of generating a user profile is set forth. In step, an application is downloaded or saved within the user device. A user identifier is generated in step. The user identifier may be generated by the user as a login name. Likewise, a password may also be saved to allow login into the device. A user identifier may be randomly generated by an alphanumeric identifier generated at the user device to identify the user device during operation. Thus, stepmay have more than one purpose generating login credentials and for identifying the user device during operation.

814 In step, personal data is stored within the user profile. Personal data such as name, address and contact information may be provided as personal data. Personal data may also include user characteristics such as body type, medical conditions, allergies, food preferences, religious preferences and the like.

816 818 In step, personal preferences may also be asked including different brands and the like. In step, various permissions may be obtained. Permissions, such as access to a camera, communicating through Bluetooth or other short range wireless signals and the like may be provided.

820 812 816 In step, a user profile is formed with the data provided in steps-. The user may provide various data to the user device through the user interface. Biometric data and the like may also be provided through the user interface, the camera or the like. In a mobile device, the facial recognition and fingerprint recognition may already be present on the device and thus may be allowed to enable the application.

822 In step, the user profile may also be based on the products purchased. That is, the user profile may be stored on an off-premises device, an on-premises device or on the mobile device. In all situations, the user profile may be updated based upon the products purchased.

824 826 In step, the dwell time and products around the user when traversing a premises may be recorded. That is, the products around a location where the user device is located for a specific amount of time or dwell time may be also used to update the profile. In step, the profile may be updated based on products at the dwell location but not purchased. That is, when a user dwells at a location and purchases the product, the user profile may be updated accordingly. However, advertisements may not be provided when products were purchased after a long dwell time at a particular location.

828 58 59 1 FIG. In step, the profile may also be updated on advertisements viewed. Advertisements provided to the user device or displayed at displays, such as displayand speakerin, may be used to update the profile. This allows repeated advertisements to be prevented or delayed for some time.

820 In step, correlation may be formed between the advertisements viewed and the products purchased. When there is a correlation, the user profile may be updated. For example, when the user purchases a particular product after an advertisement, the user profile may be updated for the user liking certain products. However, when the user just purchased products, advertisements may be delayed so that interest may be generated in other products.

9 FIG. 910 30 38 Referring now to, a method for providing advertisements to a user device is set forth. In step, the user enters the premises for a first visit. Entering the premises may be detected by a ranging sensor, a cameraor both. Likewise, other sensors on the mobile device, such as a global positioning sensor, may be used to determine the entering of a premises.

912 30 38 914 916 In step, the location of the user device within the premises is tracked. The ranging sensorsand the camerasmay track the user device within the premises. Also, the user may be tracked by either the ranging sensor or the camera when one of the two is not available. Further, because the cameras exist, the tracking and identification of the user may also be performed, for example, when the user device is left in a shopping cart. When the user device is left in a shopping cart and the user walks away, the camera signals may be prioritized over the ranging signals to determine user location. In step, the user device identifier is communicated to the ranging sensor. The images are used to classify shoppers local to a camera of a known location. Ultimately, the images are used to classify user-product interactions, and update a profile based on the user-product interactions. In step, the user device identifier and the profile associated therewith are updated at the on-premises controller. The products around the user and the interactions may be used to update the profile.

918 920 922 926 928 930 When the user device is a device not logged into an application or not specifically belonging to a user such as a shopping cart this step is optional. In step, a product associated with the first location is identified. The product may be identified by correlating the location with the planogram and map of the premises. In step, the product data and the location data are communicated to the off-premises controller. In step, the user profile is updated at the off-premises controller with the product data. When the user purchases products, the user profile is updated at the off-premises controller with the purchase data. In step, the user device location being off-premises is determined. In step, an advertisement is communicated to the user device based on the updated user profile. Various types of advertisements for various locations visited may be provided. The dwell times and other journey data may be considered when determining the advertisement. In step, the user profile is updated with advertisements presented to the user profile and prevents repeat advertisements from being provided to the user device. In summary, advertisements may be provided to the user device after the user device has left the premises and may be based on prior actions and the user profile of the user.

918 938 940 58 59 940 942 944 940 916 946 948 948 930 Referring back to step, after the product is identified that is associated with a first location. Advertisements or product data may be automatically presented or requested by the user as shown in optional step. In stepa first on-premises advertisement or product data may be presented to the user device. The first on-premises advertisement or data may be presented automatically on the user device or display, such as the displayand/or speaker, within the premises which are controlled by the on-premises controller. After step, stepcorrelates the first on-line premises advertisement to purchases. The user profile is updated in stepthat shows a correlation between the purchases and the advertisement presented in step. When the user device does not belong to a user or the application is not active (as per Step), the presentation of a rewards card will allow association of the purchased products to the user and user profile. In step, when the user, along with the user device, enters the premises for a second visit, steppresents a second advertisement that is different than the first advertisement when the first advertisement does not correlate with a purchase. The point of sale device may communicate products purchased to the on-premises controller and therefore a comparison may be made between the products in the advertisements and the products purchased. Advertisements for other products may be provided when the first product was not purchased. The advertisements may be for related products. Related products may be similar products, products in the same field, or products used together with the first product. After step, stepmay be performed in which the user profile is updated with the advertisements presented to prevent repeat of advertisements.

10 FIG. 9 FIG. 32 32 316 38 30 928 948 1010 1012 1014 352 Referring now to, an example of microlocation and an advertisement is set forth on the user device. The user devicehas the displaythat is adjacent to “J-Skin Crème”. The microlocation is determined by the cameras, the ranging sensorsor both. The advertisement may be the advertisement set forth inat stepor the advertisement at step. In this example, the advertisement has a couponand product data. A “save” buttonmay also be provided on the screen so that the advertisement is saved within the memoryof the user device or together with the user profile in the on-premises controller or the off-premises controller.

11 FIG. 1110 40 12 38 1112 1114 38 1116 Referring now to, a method for updating a user profile when a user device is not used. In step, the userenters the premises. In this example, the ranging sensors are of no use because no user device is provided. Entry into the premises is determined by the cameras. In step, no user device is therefore detected because no user device is provided by the user. In step, the user locations may be tracked with the cameras. In step, the products associated with the various locations and the dwell times associated with the locations may be used to identify products relative to the planogram.

1118 1120 1122 1124 In step, the user presents a reward card with a user identifier or bar code to the point of sale device upon checkout. In step, the user purchases the products at the point of sale device. In step, the purchased products, the location and the products associated with the locations that the user traveled are associated with the user profile that is associated with the identifier card. In this manner, the user profile is updated. Optionally, if the user uses a computer or a user device off-premises or next visit on store premises, advertisements may be communicated to the user at step. Advertisements may be communicated from the off-premises location.

12 FIG. 1210 1212 1214 1216 1218 1220 Referring now to, a method of determining the user location using cameras and the RF sensors is set forth. In step, the user enters the premises. In step, the user device communicates an identifier from the user device to the ranging sensors. In step, the ranging signals are communicated from the ranging sensors to the on-premises controller. In step, the identifier signals are communicated from the user device to the on-premises controller through the Wi-Fi. In step, the identifier signals are matched to the user profile at the on-premises controller. In step, a first set of locations based on the ranging signals are determined as an RF path.

1222 1224 1226 1220 1228 1230 In step, the camera signals are used to detect objects at various locations corresponding to users. The cameras signals may be used to determine and track the users. In step, the user moves into different object locations to form a journey. That is, the camera signals, whether video or still images, have objects that move and therefore various objects may be considered as being users or user devices. In step, the first set of object locations is compared to the object locations of the camera. After stepand, the user is identified by comparing the locations in the camera path to the locations in the RF path. That is, the object locations from the camera signals are compared to the object locations from the RF sensors. By linking the camera path and the RF path, the user device can be precisely identified along with the user. This may be specifically important when an unconnected user device is used. Although the user device is not connected per se, Bluetooth may be enabled. Bluetooth® may broadcast with identifiers that change over time. Therefore, as the user device moves the cameras may track the device at the transition from one identifier to another to allow continual identification. In step, the user profile may then be updated with orientation data from the camera images and/or the sensors provided on the user device. This will allow the system to identify precisely the products that may interest the user of the user device.

13 FIG. 12 FIG. 12 FIG. 1310 1312 36 1316 1318 Referring now to, the data frommay also be used to generate a modification of a planogram. In step, the steps ofare performed for a plurality of user devices. That is, the precise paths that the user devices traverses through the store as well as the dwell time to determine the products that the user is interested in as well as the products that are purchased. In step, the journey associated with each of the user devices is saved in the on-premises controller. The dwell times and various locations are also determined. In step, the products associated with the journey of each of the user devices are determined. In step, purchases are also determined.

1320 1322 1324 High traffic areas are determined from the above data in step. That is, the purchases, the journey, the products associated with the journey as determined by the dwell times is used to identify high traffic areas with the premises. Based on the aggregate data from many users in step, stepgenerates premises modifications of a planogram.

14 FIG. 1410 1412 1410 1412 1414 1416 1418 1420 1422 Referring now to, a planogramis illustrated with a perspective viewof the premises layout. A portion of the planogramis illustrated in the perspective view. In this example, various labels such as “less travelled” atand instruction atfor swapping a product location, and other indicators such as the summer traffic at step, seasonal products atand low traffic atare identified. The labels on the screen display of the perspective view provide the premises operators suggestions as to how to change and make modifications to a planogram. Of course, lists of changes in text form may also be generated instead of or in addition to the screen display. The planogram is a diagram or model that indicates the placement of retail products on shelves. The goal of the retailer is to maximize sales with a planogram and rearrange products as needed.

15 FIG. 1510 1512 1514 1516 1518 1518 1520 1518 1520 1520 1520 1520 1520 1520 1522 1522 1518 1522 1524 448 Referring now to, a method of providing products through a secured display case is set forth. In step, a key is generated at a key generator. The key generator, as described above, may be located in the user device, the on-premises controller or the off-premises controller. In step, the key is associated with a user profile. The key may be requested prior to entering a premises or may be located within a premises. The key may also be established at the time the user profile is generated when first signing into the application. In step, a key is stored in the user device to form a user device key. In step, the key is communicated to the on-premises controller to form a stored key. These steps may be reversed when the key is generated at the on-premises generator and stored in the user device. In step, the location of the user device is determined. As mentioned above, cameras and the ranging sensors may be used alone or together to determine the precise location of the user device. After step, optional steps may be performed. That is, access to a product may be requested in stepA. After step, an optional stepB may also be performed instead ofA. That is, stepA determines if the user is within a close proximity to the display case. Close proximity may be within or less than a predetermined distance such as 2 feet. Of course, the distance threshold of 2 feet may be increased or decreased depending upon the design considerations. When the user is not within close proximity (less than a distance threshold), stepis repeated. StepsA andB are optional steps after which stepis performed. Stepmay also be performed directly after step. In step, the user device key is communicated to a receiver or transceiver of a display device. In step, the user device key is compared to the stored key at the display device. Optionally, this step may be eliminated if the user device key is communicated to the on-premises controller for comparison at the comparison controller.

1526 356 368 1528 1524 1526 1530 1530 1532 3 FIG. In step, there may be restrictions on accessing the display case. As mentioned above, there may be an age threshold, or an authentication level threshold associated with a display case. The age threshold may be limited for products such as state restricted products like alcohol or cigarettes. The status memoryand the age memoryofmay be used for the age and status of the user which is stored in the user profile. When the age is not greater than the age threshold or the authentication level is not greater than a status threshold, access may be denied in step. After stepor when stepin which the age is greater than an age threshold or an authentication level is greater than a status threshold, the system determines the presence of untrusted devices nearby and may initiate a time delay or wait until there are no or few untrusted devices nearby. A user device may become untrustworthy for many reasons including leaving cases open and returning products too many times. Stepobtains the inventories prior to opening the display case. The display case inventory may be obtained when the last opening has taken place or after a restocking as well. The first display case inventory is obtained prior to opening at step. A confirmation query may be presented to the user through the user device to confirm that the intent is to open the display case and that the user is responsible for all items removed. For example, intent may be established when a phone is touching or within a small distance from the handle. An RF signal such as Bluetooth® or a near filed communication may be exchanged between the user device and the display case. To prevent theft, the application controlling the shopping experience may be required to be active. If the user device enters a sleep mode, the communication signal for providing intent may be stopped. A reminder may be presented to close the door and make sure it is secure before leaving the display case. In step, the display case is unlatched to allow opening of a door to access the products therein.

1536 746 1536 1536 1538 1540 1536 1542 1542 1542 1544 1546 1550 1546 1552 1552 In step, a second display case inventory is determined after the display case is closed. That is, a product identifier, such as an RF device, may be used to read the tags before and after to obtain the inventories. In step, an inventory difference is provided. After step, stepmay add the inventory difference items to a virtual shopping cart. In step, the account associated with the user device may be billed. The payment terms may be provided on the establishment of a user profile. Of course, other payment methods may be obtained or queried during this process. Alternatively, after step, stepis performed in which the user “checks out” of the premises. In step, the user or a cashier scans the products at a point-of-sale device on the premises. After step, stepcompares the inventory difference to the scanned items. In step, if the inventory difference is different than the number of scanned items, an alert signal is generated in step. An employee may provide assistance based upon the alert signal. In step, when the inventory difference is not different than the scanned items, the checkout process is finished in step. In step, the completion of the checkout process allows a payment to be made to the store for the items that are being purchased.

16 FIG. 1612 1612 1614 Referring now to, an example of a screen display for a point-of-sale device is illustrated. Because the identity of the user and user profile is known, a customized messagemay be provided. The user may be identified by communicating signals to the point-of-sale device using short range communications such as near-flied communications or scanning a reward card (virtual or physical), or identity card that identifies a user. In this example, the point-of-sale system knows that “Andrew” is the user in the personalized message. The point-of-sale device may instruct the user to perform various functions including starting the scanning process by the message.

17 FIG. 1710 1712 1714 1442 1444 1716 1718 456 1720 62 Referring now to, a method for routing a customer or user to a product is set forth. In step, a user application is enabled. In step, the product desired is searched for within the application. In step, the location of the user device on the premises is determined. The location of the product may be determined from the planogram controllerand the map. In step, the product having the location associated therewith is selected by the user from the app. In step, the router controllermay determine a route from the user device to a product location within the premises. In step, the route may be communicated to the user device through the networkand the route may be displayed either audibly or visually to the user so that the products may be easily located. Aisle numbers, bin numbers and shelf numbers may be provided at the end of the turn-by-turn routing.

18 FIG. 4 FIG. 1810 1810 1812 1814 1816 1818 1820 458 1822 Referring now to, the employee user device may be directed to a user in need. In step, the user device application is enabled at step. In step, the location of the user device on the premises is determined. As mentioned, the cameras and the ranging sensors may be used to determine the location of the user devices. In step, knowledge data may be associated with an employee device. That is, certain employees may have certain knowledge in certain categories. For example, plumbing, electrical, face creams, fresh produce and the like may be selected by an employee as areas of expertise. In step, the location of an employee device on the premises is determined. As mentioned above, the employee device may be located in a similar manner to the location of the user device. In step, a request for help may be generated at the user interface of the user device using the application. In step, the system may select an employee at the employee selector controllerof. That is, a match to the type of request may optionally be generated so that a proper employee may be deployed to help the user. In step, directions are communicated to the employee device so that the employee device can reach the user device. That is, step by step instructions may be provided so that the employee device may quickly and promptly reach the employee device. As the user device moves, the instructions may also be changed because the locations of both the user device and the employee device are changing. That is, updated instructions or directions from the employee device to the user device when the customer user device moves.

1816 1826 1828 1820 Referring back to step, another use for determining positions of users and employees is set forth. In step, the transitions between zones in a store may be tracked. in step, store operation may be adjusted. In one example, store operation may communicate to an employee device of an assignment in a different location or zone of the store as in step. Other store operations adjustments may include engaging a shopper nearby locating the shopper and adjusting the total amount of employees in an area.

19 FIG. 1910 1912 1914 1916 1918 1918 1920 1916 1922 1924 Referring now to, a method for modifying the customer package goods is set forth. In step, the user location on the premises is determined. In step, marketing choices may be determined. In step, the effectiveness of the marketing choices may be determined for effectiveness. In step, the timing of the purchases may also be determined. In step, the user profiles are determined. The user profiles for many user devices and therefore users are determined in step. In step, the data collected may be communicated to the consumer package goods suppliers. That is, the effectiveness of the various marketing campaigns, whether direct displays at the premises or on the user device, may be determined for effectiveness by determining whether the products were purchased in step. The user profiles are updated continually based upon the purchases and various advertisements selected. In step, the advertisements and the type of advertising used by a consumer package goods supplier may be modified based upon the data collected from the premises and various premises within a retail chain. In step, the product availability may also be modified. That is, when consumers are not purchasing certain products, the products may not be continued to be supplied to the various retailers for use on the various premises.

20 FIG. 20 30 1010 1012 1014 20 Referring now to, a diagram of shelving unitsis illustrated with respect to three users and four ranging sensors. In this example, the ranging sensors are labeled “A”, “B”, “C” and “D”. The data for a user identifier is set forth as “device ID” in the data boxes,and. In this example, the distances to each ranging sensor A, B, C and D are illustrated. The different distances allow the on-premises controller to relatively precisely determine a location. The person “Ace” is identified as device identifier “100” and is in a zone that is classified as hot sauce. The user “Pop” identified as device identifier “200” and is identified in the zone referred to as “pasta”. The device identifier “YYZ” is positioned at the zone “instant rice”. These methods may also be supplemented by the camera image which may also provide the orientation. Of course, the other sensors described above with respect to the user device may be used to provide the orientation of the user relative to the shelving units.

Example embodiments are provided so that this disclosure will be thorough and will fully convey the scope to those who are skilled in the art. Numerous specific details are set forth such as examples of specific components, devices, and methods, to provide a thorough understanding of embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms and that neither should be construed to limit the scope of the disclosure. In some example embodiments, well-known processes, well-known device structures, and well-known technologies are not described in detail.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms “comprises,” “comprising,” “including,” and “having,” are inclusive and therefore specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as an order of performance. It is also to be understood that additional or alternative steps may be employed.

When an element or layer is referred to as being “on,” “engaged to,” “connected to,” or “coupled to” another element or layer, it may be directly on, engaged, connected or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly engaged to,” “directly connected to,” or “directly coupled to” another element or layer, there may be no intervening elements or layers present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., “between” versus “directly between,” “adjacent” versus “directly adjacent,” etc.). As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

Although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms may be only used to distinguish one element, component, region, layer or section from another region, layer or section. Terms such as “first,” “second,” and other numerical terms when used herein do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the example embodiments.

Spatially relative terms, such as “inner,” “outer,” “beneath,” “below,” “lower,” “above,” “upper,” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. Spatially relative terms may be intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below” or “beneath” other elements or features would then be oriented “above” the other elements or features. Thus, the example term “below” can encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.