Railroad crossing warning system for enhanced route planning based on spectrogram analysis

None

The present disclosure is in the field of railroad transportation. More particularly, the present disclosure provides systems and methods of detecting sounds associated with warning bells at railroad crossings that indicate passing trains and based thereon advising nearby motorists via wireless notification of potential for traffic congestion and delay.

Railroad transportation, for moving both passengers and freight, are essential to the economic and social well-being of developed countries. Freight trains transport a major portion of US goods daily. This has inadvertently led to problems with blocked railroad crossings, for extensive periods of time in many cases. Railroads used by the freight industry often travel through urban areas and cause extensive automobile traffic delays due to their length and slower speeds. Freight trains could stretch up to three miles and are operated at very low speeds through residential areas. The issues of railroad crossing safety and traffic congestion have reached high levels of the US federal government but have failed to receive action. Further, there have been documented cases of delayed emergency responses.

Although advanced route planning technology is available, such technology does not provide information relating to the state of railroad crossings. This information is not shared by railroad operators. Rail companies have state-of-the-art monitoring systems that are not available for public or city municipality use.

Previous implementations describe devices using laser or vibration to detect incoming trains and may require cooperation of the railroad involved including using components owned by the rail company. No previous implementation independently identifies the state of a crossing.

Most railroad crossings now have protection in the form of automatic warning devices such as flashing lights, warning sounds, and barriers or gates. Crossing gates and electronic bells are activated simultaneously before the arrival of the train and stay active for several minutes after the train has passed. But the problem of automobile traffic congestion caused by long and slow freight trains remains.

Systems and methods described herein provide for observing the open or closed state of a railroad crossing and making that information available to motor vehicle operators and others in near real time via a mobile application. Ambient sound is captured, converted into frequencies and then depicted or plotted as a spectrogram. The spectrogram is analyzed in its entirety to detect specific sound patterns by a proprietary algorithm. The algorithm was developed by capturing hundreds of hours of sound samples to train a neural network. It was trained to a point where it could detect bell sounds with 99% or higher accuracy. Electronic bells that are typically mounted on masts supporting gates or tall poles near crossings are activated by the railroad as a train is approaching. Systems provided herein are programmed to detect when the electronic bell begins and stops sounding.

Systems and methods of the present disclosure have two primary objectives: 1) Quickly and accurately determine the state of railroad crossing gate, either open or closed; and 2) Communicate with the mobile application in proximate and distant vehicles as quickly as possible.

As a train approaches a crossing, the crossing gate closes and the bell begins to sound. Electronic bells used by railroads have an array of frequencies. A device that may be referred to as the Field-Device (FD) is deployed in the vicinity of a crossing and listens continuously using a digital or an analog microphone. The surrounding noise is captured and processed by a microprocessor that may be dual core and an application executing thereon. When the bell begins alarming such that a train is approaching, an algorithm or formula mentioned previously is activated by the application. The algorithm or formula detects the specific sound patterns on the spectrogram and orders a communication module to update status of the crossing to “closed” in a mobile application executing on motorist devices. The same algorithm also detects when the bell has stopped alarming to indicate reopening of the crossing. The microprocessor then orders the communication module to send updates to motorists' devices changing the crossing status to “open”.

The application, including at the least one algorithm, periodically samples ambient sound, for example every 3 seconds using an analog or a digital mic (I2S protocol), performs the appropriate version of the Fast Fourier Transform on the incoming data to create a spectrogram. The components of the spectrogram are then analyzed by a proprietary algorithm developed using the neural network and machine learning. This analysis of the spectrogram ensures accurate and fast detection of the state of the gate.

The device provided herein is equipped with a solar panel that powers all components of an Integrated Circuit (IC) when optimal sunlight is available. The solar panel also charges the onboard Lithium-ion battery during the optimal sunlight hours. The Li-ion battery provides power when the solar panel cannot provide power. The device is energy independent. Power needs of systems provided herein may be met by the structure described above and may reduce or eliminate a need for maintenance.

The lightweight design and small footprint of the device allows it to be mounted conveniently in the vicinity of railroad crossing. The device is not in most embodiments mounted on any of the active components of the railroad system owned by the railroad company.

Other than listening for sounds emitted by the bell at crossings, systems and methods provided herein do not control or take input from any railroad crossing components or other components controlled by the railroad. Further, no action or participation by the railroad is required or involved. The device provided herein which contains the microprocessor, microphone, and cellular module may be mounted, for example, on a street sign, on a sign board, or on a residential awning in the vicinity of the railroad crossing. The detection and warning methods provided herein are fully independent of the railroad.

The field deployed detection and reporting device contains the microphone. The device also contains the microprocessor and the communication module.

The device may share data with local public transportation authorities. Buses equipped with GPS functionality may use functionality provided to reduce delays and improve rider experience. First responders' response time may be greatly improved by having this information in advance. Rideshare organizations such as Lyft and Uber may also benefit from systems and methods provided herein.

In an embodiment, a network of interconnected field devices such as that provided herein may be built. Existing backend server functionalities to determine crossing status when one of the devices in the chain malfunctions may be used. The data gathered by such device network can also be used to train a Neural Network which, when used in conjunction with the Artificial Intelligence (AI), will open numerous possibilities to either directly improve the system described here or provide add-on services in the future.

As part of getting information from the point-of-action to the end user, a mobile application was developed. With its user-friendly interface, it shows the state of the gate with color coded rings. These rings are superimposed onto the map of the area of focus. The overall system is designed to expand the capability nationwide. The app can be updated to include time approximation as data becomes available post deployment.

1 FIG. 1 FIG. 100 Turning to the figures,is a diagram of a system of gate tracking as provided herein.illustrates components and interactions of a systemof railroad crossing warning for enhanced route planning.

100 102 104 102 104 104 1 FIG. The systemcomprises a field deployed detection and reporting deviceand a field deployed detection and reporting applicationexecuting thereon, referred to hereafter for brevity as the deviceand the application, respectively. The applicationincludes at least one algorithm that is not shown in.

100 106 108 110 100 112 114 116 118 118 116 114 116 100 116 102 104 a n a n a n a n a n a n a n a n The systemalso includes a microprocessor, a microphone, and a communication modulewhich in some embodiments may be referred to as a cellular module. The systemfurther includes a backend server, motor vehicles-, mobile devices-, and mobile applications-. The mobile applications-execute on the mobile devices-which are carried in motor vehicles-that may travel near railroad crossings and be subject to delays by passing trains. Mobile devices-may be associated with drivers or other users participating in a subscription program to receive notifications and other services described herein provided by the system. In some embodiments, those subscribing mobile devices-may be identified as “selected” or otherwise “associated with” the deployed detection and reporting deviceand a field deployed detection and reporting application.

1 FIG. 100 120 120 122 124 126 128 Also illustrated for discussion purposes inbut not directly provided by the systemare a railroad crossing and vicinitywhich may be referred to for brevity as the crossing, an approaching train, a train passing through crossing, a receding train, and a warning bell, all components provided by a railroad company in most embodiments.

122 124 126 120 120 128 108 128 106 104 104 102 110 112 118 118 118 100 a n a n a n 2 FIG. 3 FIG. As approaching trains, trains passing through crossing, and receding trains, i.e., trains leaving the crossing, are in effect at a crossing, the warning bellis sounded by the railroad company to warn motorists. A barrier may also be lowered to physically obstruct vehicles from crossing tracks. The microphoneis receiving and passing along sounds of the warning belland all other proximate sounds to the microprocessor. The application, as noted, samples ambient sound periodically, performs Fast Fourier Transform on the captured data to create a spectrogram. The spectrogram is then algorithmically analyzed for data suggesting approaching of a railroad train. As described in greater detail above, as long as the results of the analysis meets criteria contained in the application, the device, via the cellular module, notifies the backend serverwhich transmits messages to the mobile applications-. Motorists and others who have downloaded the mobile application-on their phones have this information readily available when the application is launched. Availability of information as well as notifications provided by the mobile application-is not dependent on users' geographic location. Components inandare indexed to components provided by the system.

2 FIG. 2 FIG. 2 FIG. 202 212 218 212 110 100 218 206 208 210 a a shows communication between the deviceand backend serversupporting the mobile application. The software interactions between the components are illustrated in, particularly the software configuration of the backend server. What is depicted is equivalent to the communication moduleprovided by the system.additionally shows an exemplary mobile application or app, an exemplary microprocessor, an exemplary microphone, and an exemplary communication module or module.

3 FIG. 302 312 318 312 302 302 302 illustrates multiple devices, shown as Gate Tracking Devices, transmitting through the backend serverto mobile applications. In embodiments, a single backend servercan receive messaging from a plurality of devices. If these devicesare located at railroad crossings on the same rail line in geographical succession, their reported information may be used in determining if a railroad train has stopped, its direction, or if one of the deviceshas malfunctioned.

In an embodiment, a system for detection of trains at railroad crossings is provided comprising a field-deployed detection and reporting device. The device comprises a microphone, a communication module, a microprocessor, an application executing on the microprocessor. The application receives periodically generated samples of ambient sound captured by the microphone and performs Fast Fourier Transformation on data associated with the generated samples. The application also creates a spectrogram based on the output of the performed Fast Fourier Transformation, analyzes components of the spectrogram using at least a proprietary algorithm, and transmits, based on the analysis, a first message via the communication module.

The device is situated proximate a railroad crossing. The ambient sound is generated by a warning bell sounded at the railroad crossing.

The warning bell is sounded to indicate presence of a railroad train. The first message is received by a backend server that issues a first broadcast based on receipt of the first message.

The broadcast is directed to selected wireless devices and warns at least of temporary closing of the railroad crossing. The application further determines that the ambient sound associated with the warning bell discontinues.

Based on the determination, the application sends a second message via the module to the backend server. Based on receipt of the second message, the backend server issues a second broadcast, the second broadcast indicating that the railroad crossing is reopened.

In another embodiment, a system for reducing vehicle traffic delays at railroad crossings is provided. The system comprises a computer and an application executing on the computer that processes a series of sounds received by a microphone proximate the computer. The application also performs Fast Fourier Transform (FFT) analysis on data associated with the sampled ambient sounds. The application also creates and analyzes, via a proprietary algorithm, a spectrogram based on the data. The application also instructs a server, via a cellular module and based on the analysis, to transmit a broadcast.

The computer is situated proximate a railroad crossing. The sampled ambient sounds are generated by an alarm bell at the railroad crossing.

The analysis of at least the spectrogram indicates that a railroad train is approaching the crossing. The broadcast is directed to selected wireless devices.

In yet another embodiment, a method of reducing vehicle traffic delays at railroad crossings is provided. The method comprises a computer equipped with a communication module receiving from a proximate microphone a plurality of electronic signals representing ambient sounds captured by the microphone. The method also comprises the computer, based on performance of a fast Fourier transform on data associated with the captured sounds, creating a spectrogram representing the data. The method also comprises the computer, based on analysis of the spectrogram by at least one proprietary algorithm, transmitting via the module a first message to a backend server.

The method also comprises the backend server issuing a broadcast to wireless devices associated with the computer. The method also comprises the computer, based on the analysis, determining that the ambient sounds indicate a railroad train is present at a railroad crossing proximate the microphone.

The ambient sounds are associated with a warning bell situated at the railroad crossing. The method also comprises the broadcast warning the wireless devices of a closed status of the railroad crossing. The method also comprises the computer subsequently transmitting via the module a second message to the backend server, the second message advising of an open status of the railroad crossing, the second message based on continued analysis of ambient sounds proximate captured by the microphone.