Information Processing System, Information Processing Method, and Program

The present invention relates to an information processing system, an information processing method, and a program.

Patent Literature 1 evaluates cardiovascular disease risk.

Patent Literature 1: Japanese Unexamined Patent Application Publication No. 2022-136784

It is difficult to determine a patient's subjective condition such as mental health.

The present invention has been made in view of such background and aims to provide a technology capable of supporting the differentiation of mental disorders.

The principal invention for solving the above problem is an information processing system comprising: an answer input unit configured to accept an answer from a patient to a question for differentiating a mental disorder; a video image acquisition unit configured to acquire a video image capturing the patient; a biometric reaction detection unit configured to analyze the video image to detect a change in biometric reaction of the patient; and an estimation unit configured to estimate the mental disorder that the patient suffers from by inputting the accepted answer and the detected change in biometric reaction into a learning model, wherein the learning model is trained on the answer, the change in biometric reaction, and the mental disorder.

Other problems disclosed in this application and their solutions will be made clear in the embodiment section and drawings.

According to the present invention, it is possible to support the differentiation of mental disorders.

The following describes an information processing system according to an embodiment of the present invention. The information processing system of this embodiment supports the differential diagnosis of mental disorders (depression, bipolar disorder, obsessive-compulsive disorder, sleep disorder, eating disorder, alcohol dependence, adjustment disorder, autism spectrum disorder, attention deficit hyperactivity disorder, schizophrenia, dementia, developmental disorder, panic disorder, PTSD, etc.). The information processing system of this embodiment differentiates mental disorders (or normal condition) based on a patient's answers to questions for differentiating mental disorders (tests such as QIDS, DSM-IV, DSM-5, ADOS-2, ASRS, EAT-26, MMSE, etc.) and the analysis results of a video image capturing the patient's conversation behavior (daily conversation).

Conversations in medical settings can be broadly divided into specialized conversations for examination and diagnosis (conversations conducted as part of medical practice, such as confirming symptoms, explaining test results, discussing treatment plans) and daily conversational exchanges (casual conversations in waiting rooms, small talk about weather or family updates before and after examinations, natural dialogues not for diagnostic purposes). In this embodiment, daily conversation refers to linguistic communication as a social interaction that naturally occurs between people present, not conducted for specific duties or purposes. Daily conversation is characterized by low purposefulness (not primarily aimed at specific problem-solving or information gathering), spontaneity (not planned), bidirectionality (mutual interaction rather than one-way information transmission), and functions to maintain/build social relationships. In other words, among linguistic communications occurring in the same medical setting, conversations for medical interviews or diagnoses are excluded from “daily conversation,” while casual conversations exchanged between examinations are included in “daily conversation.” Note that daily conversation includes situations where the patient is speaking one-sidedly (medical professionals or agents are mainly listening to the patient).

Additionally, the information processing system of this embodiment accepts input of differentiation results from a physician (including non-physician medical professionals; the same applies hereinafter) and determines the validity of the physician's differentiation results by comparing them with the differentiation results based on the answers and video image.

1 FIG. 2 2 1 3 shows an example of the overall configuration of the information processing system. The information processing system of this embodiment includes a management server. The management serveris connected to a physician terminaland a patient terminalvia a communication network to enable communication. The communication network is, for example, the Internet, and is constructed using public telephone networks, mobile phone networks, wireless communication channels, Ethernet (registered trademark), etc.

1 1 The physician terminalis a computer operated by a physician. The physician terminalcan be, for example, a smartphone, a tablet computer, a personal computer, etc.

3 3 The patient terminalis a computer operated by a patient. The patient terminalcan be, for example, a smartphone, a tablet computer, a personal computer, etc.

2 The management servermay be a general-purpose computer such as a workstation or a personal computer, or it may be logically implemented through cloud computing.

2 FIG. 2 2 201 202 203 204 205 206 203 204 205 206 2 201 203 202 2 202 203 shows an example of the hardware configuration of the management server. Note that the illustrated configuration is an example and may have configurations other than this. The management serverincludes a CPU, memory, storage device, communication interface, input device, and output device. The storage devicestores various data and programs, and is, for example, a hard disk drive, solid-state drive, flash memory, etc. The communication interfaceis an interface for connecting to a communication network, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or RS232C connector for serial communication, etc. The input deviceinputs data, and is, for example, a keyboard, mouse, touch panel, button, microphone, etc. The output deviceoutputs data, and is, for example, a display, printer, speaker, etc. Each functional unit of the management serverdescribed later is realized by the CPUreading and executing a program stored in the storage deviceinto the memory, and each storage unit of the management serveris realized as part of the storage area provided by the memoryand the storage device.

3 FIG. 2 2 211 212 213 214 215 216 217 231 shows an example of the software configuration of the management server. The management serverincludes an answer input unit, a video image acquisition unit, a biometric reaction detection unit, an estimation unit, a diagnosis result input unit, a misdiagnosis possibility determination unit, a daily conversation processing unit, and a learning model storage unit.

231 231 231 2 2 The learning model storage unitstores a learning model trained by machine learning for estimating mental disorders (hereinafter referred to as a “disorder model”). The disorder model stored in the learning model storage unitmay be, for example, one trained by machine learning using training data consisting of a patient's answers to questions for differentiating mental disorders, changes in the patient's biometric reactions, and the mental disorder the patient suffers from (or the fact that the patient is healthy). The disorder model can be a classifier. The disorder model may also be a generator (large language model) with fine-tuning. The learning model storage unitmay be provided by an external server rather than by the management server, allowing the management serverto access the external server.

231 2 The learning model storage unitmay also store a learning model trained by machine learning for determining a physician's diagnosis result (hereinafter referred to as a “determination model”). The determination model can be created by machine learning using training data consisting of the estimation result of a mental disorder using the disorder model, the diagnosis result by a physician, and the correctness of that diagnosis result. The determination model may also be created using training data consisting of a patient's answers to questions for differentiating mental disorders, changes in the patient's biometric reactions, and the diagnosis result by a physician. Also, attributes of the physician (which may be information identifying the physician, or the physician's age group, years of experience, main specialty, etc.) may be given as features to the determination model. In this case, the management servermay include a physician information storage unit that manages physician attributes, and during inference, read the attributes of the diagnosing physician from the physician information storage unit and provide them to the determination model.

211 211 1 3 The answer input unitaccepts input of a patient's answer to a question for differentiating a mental disorder. The answer input unitmay accept input of a patient's answer from a physician (receive the answer from the physician terminal), or it may present questions to the patient and directly accept input of answers from the patient (receive answers from the patient terminal).

212 212 217 212 3 212 1 1 1 2 The video image acquisition unitacquires a video image capturing the patient. The video image is assumed to capture the patient engaged in daily conversation. The video image acquisition unitcan acquire a video image capturing the conversation between the patient and the daily conversation processing unitdescribed later. The video image acquisition unitcan receive a video image of the patient's behavior captured by the patient terminalusing a camera. The video image acquisition unitmay also acquire a video image capturing the patient's conversation behavior from the physician terminal. In this case, the physician terminalmay perform the recording, or a video file recorded by a separate camera may be sent from the physician terminalto the management server.

213 The biometric reaction detection unitanalyzes the video image to detect changes in the patient's biometric reactions.

213 213 213 For example, the biometric reaction detection unitcan separate the video image into a set of images (collection of frame images) and audio, and analyze changes in biometric reactions from each. For example, the biometric reaction detection unitcan analyze the user's facial image using frame images separated from the video image to analyze changes in biometric reactions related to at least one of facial expressions, gaze, pulse, and facial movements. Also, the biometric reaction detection unitcan analyze the audio separated from the video image to analyze changes in biometric reactions related to at least one of the user's speech content and voice quality.

213 When a person's emotions change, they appear as changes in biometric reactions such as facial expressions, gaze, pulse, facial movements, speech content, and voice quality. In this embodiment, by analyzing changes in the user's biometric reactions, changes in the user's emotions are analyzed. An example of emotions analyzed in this embodiment is the degree of pleasure/displeasure. In this embodiment, the biometric reaction detection unitcan calculate a biometric reaction index value that reflects the content of changes in biometric reactions by numerically quantifying the changes in biometric reactions according to a predetermined standard.

The analysis of changes in facial expressions is performed, for example, as follows. For each frame image, the face region is identified from the frame image, and the facial expression identified in that region is classified into multiple categories according to a pre-trained image analysis model. Then, based on the classification results, an analysis is made as to whether a positive facial expression change is occurring between consecutive frame images, whether a negative facial expression change is occurring, and how large a facial expression change is occurring, and a facial expression change index value can be calculated according to the analysis results.

213 The analysis of changes in gaze is performed, for example, as follows. For each frame image, the eye region is identified from the frame image, and the direction of both eyes is analyzed to determine where the user is looking. For example, it analyzes whether the user is looking at the displayed speaker's face, the displayed shared materials, or outside the screen. It may also analyze whether the movement of the gaze is large or small, whether the frequency of movement is high or low, etc. Changes in gaze are also related to the user's level of concentration. The biometric reaction detection unitcan calculate a gaze change index value according to the results of the gaze change analysis.

213 The analysis of changes in pulse is performed, for example, as follows. For each frame image, the face region is identified from the frame image. Then, using a pre-trained image analysis model that captures numerical values of facial color information (G of RGB), changes in G color on the face surface are analyzed. By arranging the results along the time axis, a waveform representing changes in color information is formed, and the pulse is identified from this waveform. People's pulse rates increase when they are tense and decrease when they calm down. The biometric reaction analysis unitcan calculate a pulse change index value according to the results of the pulse change analysis.

213 The analysis of changes in facial movements is performed, for example, as follows. For each frame image, the face region is identified from the frame image, and the direction of the face is analyzed to determine where the user is looking. For example, it analyzes whether the user is looking at the displayed speaker's face, the displayed shared materials, or outside the screen. It may also analyze whether the movement of the face is large or small, whether the frequency of movement is high or low, etc. The facial movement and gaze movement may be analyzed together. For example, it may analyze whether the user is looking straight at the displayed speaker's face, looking up or down at it, or looking at it from an angle. The biometric reaction analysis unitcan calculate a face direction change index value according to the results of the facial direction change analysis.

213 The analysis of speech content is performed, for example, as follows. The biometric reaction analysis unitconverts the audio of a specified time (e.g., about 30-150 seconds) into text by performing well-known speech recognition processing, and performs morphological analysis on the text to remove unnecessary words for expressing conversation, such as particles and articles. Then, the remaining words are vectorized, and an analysis is made as to whether a positive emotional change is occurring, whether a negative emotional change is occurring, and how large an emotional change is occurring, and a speech content index value can be calculated according to the analysis results.

12 The analysis of voice quality is performed, for example, as follows. The biometric reaction unit, by performing well-known voice analysis processing on the audio of a specified time (e.g., about 30-150 seconds), identifies the acoustic features of the voice. Then, based on these acoustic features, an analysis is made as to whether a positive voice quality change is occurring, whether a negative voice quality change is occurring, and how large a voice quality change is occurring, and a voice quality change index value can be calculated according to the analysis results.

213 The biometric reaction analysis unitcalculates a biometric reaction index value using at least one of the facial expression change index value, gaze change index value, pulse change index value, face direction change index value, speech content index value, and voice quality change index value calculated as described above. For example, the biometric reaction index value can be calculated by weighted calculation of the facial expression change index value, gaze change index value, pulse change index value, face direction change index value, speech content index value, and voice quality change index value.

214 214 214 231 The estimation unitestimates the patient's mental disorder. The estimation unitcan estimate the mental disorder based on the patient's answer and the change in biometric reaction (biometric reaction index value). The estimation unitcan estimate the mental disorder that the patient suffers from by inputting the accepted answer and the detected change in biometric reaction into the disorder model stored in the learning model storage unit.

214 Furthermore, the estimation unitcan determine the probability that the patient suffers from each of multiple mental disorders based on the confidence of the inference by the disorder model. Specifically, it uses the membership probabilities for each mental disorder class calculated in the output layer of the disorder model. Normally, the disorder model receives input data (patient's answers and biometric reaction index values) and probabilistically outputs which mental disorder class that data belongs to. For example, probabilities of suffering from multiple mental disorders are calculated, such as a 70% probability of depression, a 20% probability of bipolar disorder, and a 10% probability of schizophrenia. The confidence level may also be used as a “probability.” That is, the non-linear possibility of suffering may be called a probability.

214 The estimation unitcan evaluate the accuracy of the diagnosis based on the estimated probabilities of mental disorders. The reliability of the estimation result can be evaluated by the absolute value of the probability of suffering from the mental disorder with the highest probability. For example, if the probability of depression is 90% or higher, it can be determined that there is a very high possibility of depression, while if the probability of the mental disorder with the highest probability is around 50%, it can be determined that the accuracy of the diagnosis is not so high.

214 Also, the estimation unitcan suggest the possibility of comorbidities based on the estimated probabilities of mental disorders. If the probabilities of suffering from two or more mental disorders are both high, it may suggest the possibility that these disorders coexist.

215 215 1 The diagnosis result input unitaccepts the result of a differential diagnosis of a patient's mental disorder by a physician. The differential diagnosis by the physician is based on the answers above. The diagnosis result input unitcan receive the diagnosis result from the physician terminal.

216 215 216 214 215 216 214 The misdiagnosis possibility determination unitdetermines the possibility of misdiagnosis in the physician's differentiation result accepted by the diagnosis result input unit. The misdiagnosis possibility determination unitcan determine the possibility of misdiagnosis by whether the mental disorder estimation result by the estimation unitmatches the diagnosis result accepted by the diagnosis result input unit. The misdiagnosis possibility determination unitcan determine the probability of misdiagnosis according to the probabilities of mental disorders estimated by the estimation unit.

216 214 231 216 231 The misdiagnosis possibility determination unitmay infer the possibility of misdiagnosis (or the possibility of correct diagnosis) by providing the mental disorder estimation result by the estimation unitand the diagnosis result by the physician to the determination model stored in the learning model storage unit. The misdiagnosis possibility determination unitmay infer the possibility of misdiagnosis (or the possibility of correct diagnosis) by providing the patient's answers, the detected changes in biometric reactions, and the physician's diagnosis result to the determination model stored in the learning model storage unit.

216 1 216 216 3 The misdiagnosis possibility determination unitcan send the possibility of misdiagnosis to the physician terminalto warn the physician of the possibility of misdiagnosis. The misdiagnosis possibility determination unitmay also send the possibility of a physician's misdiagnosis to a terminal of a manager of the medical institution, etc. In this case, an administrator storage unit that stores information indicating an administrator, etc., for each physician can be provided, and when the possibility of misdiagnosis is above a certain value, or regardless of the degree of possibility of misdiagnosis, the administrator, etc., corresponding to the physician can be read from the administrator storage unit, and the possibility of misdiagnosis can be sent to the read administrator. The misdiagnosis possibility determination unitmay also send the possibility of misdiagnosis to the patient terminalto notify the patient.

217 217 The daily conversation processing unitconducts conversations (daily conversations) with the patient. The daily conversation processing unitcan, for example, realize natural conversation with the patient by generating appropriate responses according to the patient's speech content using a large language model. The large language model used in this embodiment is a model that can understand the context and meaning of language by learning a large amount of text data and can generate natural text like a human according to the given context. Specifically, large language models such as the GPT (Generative Pre-trained Transformer) series, BERT (Bidirectional Encoder Representations from Transformers) series, XLNet, ELMO (Embeddings from Language Models), etc., can be used.

217 3 (1) Convert the patient's speech content to text data. (2) Tokenize the converted text data to fit the input format for the large language model. (3) Input the tokenized data into the large language model and generate an appropriate response text according to the context. (4) Convert the generated response text into audio data using a speech synthesis engine and output it to the patient. The daily conversation processing unithas the large language model generate speech content to be conveyed to the patient according to the patient's speech content (which can be obtained by analyzing the audio extracted from the video image received from the patient terminal). Specifically, the following procedure is considered:

217 The following is a conversation history with a patient. Prompt example: Patient: “Hello.” System: “Hello. It's warm today, isn't it?” Patient: “Ah, yes. It might be warm.” Please consider what the system should say to the patient next. The daily conversation processing unitcan, for example, generate conversation content to be conveyed to the patient by giving a prompt to the large language model that includes the history of the patient's speech content (may be all or a specified number of the most recent ones) and an instruction to generate speech content to be conveyed to the patient according to that speech content. An example of a prompt could be as follows:

217 217 The daily conversation processing unitconducts such conversations with the patient and can obtain useful information about the patient's mental state by analyzing the content. The daily conversation processing unitcan also obtain more detailed information by analyzing not only the content of the patient's responses but also the response time, word choice in responses, speech speed, and intonation.

4 FIG. 2 is a diagram explaining the operation of the management server.

2 301 302 303 304 305 2 306 307 308 The management serverpresents questions for differentiating mental disorders to the patient (S), accepts answers from the patient to the questions (S), acquires a video image capturing the patient engaged in daily conversation (S), analyzes the acquired video image to detect changes in biometric reactions (S), and estimates the mental disorder using the learning model by providing the accepted answers and detected changes in biometric reactions (S). Additionally, the management servercan accept a physician's diagnosis result (S), determine the possibility of misdiagnosis in that diagnosis result (S), and notify the physician of the determination result (S).

As described above, the information processing system of this embodiment can differentiate a patient's mental disorder from a video image capturing the patient's behavior in addition to answers to questions. It is known that when experts observe the behavior of people suffering from mental disorders, they can infer mental disorders even without specifically asking test questions, and by adding judgment from video images, improvement in the accuracy of inference is expected. Also, even for patients who intentionally give answers different from their actual condition to questions, it is expected that the possibility of mental disorders can be correctly estimated by also performing differentiation from video images.

Also, in the information processing system of this embodiment, the possibility of misdiagnosis can be determined based on the physician's diagnosis result and the computer's inference result.

The above embodiment has been described to facilitate understanding of the present invention and is not intended to limit the interpretation of the present invention. The present invention can be changed and improved without departing from its spirit, and the present invention includes its equivalents.

2 2 For example, the processing by each functional unit of the management serverdescribed above may be executed by any functional unit. Also, a different functional unit that executes part of the processing of the functional units described above may be added. Also, the functional units of the management servermay be distributed across multiple computers.

2 Also, the information stored in each storage unit of the management servermay be stored in any storage unit. That is, the information stored in multiple storage units described above may be stored by a single storage unit, or part of the information stored in one storage unit described above may be stored by another storage unit.

By setting biometric reaction indicators that should be particularly emphasized for each type of mental disorder and focusing on analyzing changes in those indicators, more accurate diagnostic support can be provided. Below are examples of biometric reaction indicators that should be emphasized in the diagnosis of representative mental disorders.

Facial expression: Facial muscle movements are poor, and expressions of sadness or pain are often observed. Gaze: The gaze is directed downward, and the frequency of eye contact decreases. Voice quality: Voice inflection is poor, tending to be monotonous and quiet. In the diagnosis of depression, it is effective to emphasize the following biometric reaction indicators:

Facial expression: (Hypomanic state) Facial muscle movements are active, and expressions of joy or excitement are often observed. (Depressive state) Facial muscle movements are poor, and expressions of sadness or pain are often observed. Gaze: (Hypomanic state) The gaze is directed upward, and the frequency of eye contact increases. (Depressive state) The gaze is directed downward, and the frequency of eye contact decreases. Voice quality: (Hypomanic state) Voice inflection is rich, tending to be loud. (Depressive state) Voice inflection is poor, tending to be monotonous and quiet. In the diagnosis of bipolar disorder, it is effective to emphasize the following biometric reaction indicators:

Facial expression: Due to flattening of affect, changes in facial expressions become poor. Gaze: The gaze may fixate on one point or, conversely, may not settle. Speech content: Disjointed content reflecting decreased reality testing is common. In the diagnosis of schizophrenia, it is effective to emphasize the following biometric reaction indicators:

Pulse: Marked tachycardia is observed during panic attacks. Facial expression: Expressions of strong fear are observed during panic attacks. Voice quality: During panic attacks, the voice tends to tremble and words get stuck. In the diagnosis of panic disorder, it is effective to emphasize the following biometric reaction indicators:

Speech content: Statements about time and place become inaccurate, reflecting memory disorders and disorientation. Voice quality: Speech rate tends to slow down, and voice volume tends to decrease. Facial expression: Due to flattening of affect, changes in facial expressions become poor. In the diagnosis of dementia, it is effective to emphasize the following biometric reaction indicators:

Note that the biometric reaction indicators exemplified above are merely representative and the biometric reaction indicators themselves are just examples.

Patient attributes such as age and gender can be considered when estimating mental disorders or determining the possibility of misdiagnosis. Examples of attribute information that should be considered during estimation and determination are shown below.

The prevalence and manifestation of symptoms of mental disorders may differ by age. For example, dementia is common in the elderly and rare in young people. Also, autism spectrum disorder in childhood may show behavioral characteristics different from those in adults. Therefore, by considering the patient's age, it is possible to evaluate using diagnostic criteria more appropriate for that age group.

214 231 The estimation unitmay provide the patient's age as one of the input information to the disorder model stored in the learning model storage unit. This enables estimation of mental disorders according to age.

216 231 The misdiagnosis possibility determination unitmay provide the patient's age as one of the input information to the determination model stored in the learning model storage unit. This enables determination of the possibility of misdiagnosis considering age.

It is known that some mental disorders show gender differences in prevalence and symptoms. For example, the lifetime prevalence of depression is higher in women, and the lifetime prevalence of alcohol dependence is higher in men. Also, eating disorders are more common in women. Therefore, by considering the patient's gender, it is possible to make diagnoses that take into account symptoms and prevalence characteristic to gender.

214 231 The estimation unitmay provide the patient's gender as one of the input information to the disorder model stored in the learning model storage unit. This enables estimation of mental disorders according to gender.

216 231 The misdiagnosis possibility determination unitmay provide the patient's gender as one of the input information to the determination model stored in the learning model storage unit. This enables determination of the possibility of misdiagnosis considering gender.

The patient's history of mental disorders or physical illnesses has a significant impact on the interpretation and diagnosis of current symptoms. For example, if a patient with a history of depression again presents with depressive symptoms, it is more likely to be a recurrence of depression rather than a simple stress response. Also, a history of thyroid disease or cerebrovascular disease can cause psychiatric symptoms. Therefore, by considering the patient's medical history, more appropriate diagnoses can be made.

214 231 The estimation unitmay provide the patient's medical history as one of the input information to the disorder model stored in the learning model storage unit. This enables estimation of mental disorders based on medical history.

216 231 The misdiagnosis possibility determination unitmay provide the patient's medical history as one of the input information to the determination model stored in the learning model storage unit. This enables determination of the possibility of misdiagnosis considering medical history.

214 216 The information processing system of this embodiment may include an attribute information storage unit that manages the patient's attributes such as age, gender, and medical history. The estimation unitand the misdiagnosis possibility determination unitcan read necessary attribute information from the attribute information storage unit and use it for estimation and determination processing.

Also, attribute information may be input by physicians or patients, or it may be automatically extracted from medical questionnaires or medical records.

216 When the misdiagnosis possibility determined by the misdiagnosis possibility determination unitis high, various methods can be used to alert physicians and patients.

216 1 The misdiagnosis possibility determination unitmay display an alert message on the physician terminalwhen it determines that the possibility of misdiagnosis is high. The alert message can include the fact that there is a high possibility of misdiagnosis and the reason for it (e.g., the disorder estimated from the patient's answers and biometric reactions differs from the physician's diagnosis). The alert message may also include wording that encourages reconsideration or suggests additional tests or interviews.

216 The misdiagnosis possibility determination unitmay automatically include the fact that there is a high possibility of misdiagnosis in the diagnosis report created by the physician when it determines that the possibility of misdiagnosis is high. This allows the physician to recognize the possibility of misdiagnosis when reviewing the diagnosis report. The misdiagnosis possibility information included in the diagnosis report can also be used for determining subsequent treatment policies and sharing information with other medical staff.

216 The misdiagnosis possibility determination unitmay send an alert message to the administrator of the medical institution or the supervisor of the department when it determines that the possibility of misdiagnosis is high. This enables management of diagnosis quality and, if necessary, guidance and support for the physician. The alert message can include the fact that there is a high possibility of misdiagnosis, the reason for it, and information such as the name of the relevant physician and patient.

216 3 The misdiagnosis possibility determination unitmay display an alert message on the patient terminalwhen it determines that the possibility of misdiagnosis is high. The alert message can convey that the current diagnosis result needs reconsideration and include content encouraging additional tests or interviews with the physician. However, careful consideration must be given to the content and expression of the message to avoid causing anxiety to the patient.

216 The misdiagnosis possibility determination unitmay record and analyze the history of alerts that have occurred. The alert history can include information such as the date and time the alert occurred, the names of the physicians and patients involved, and the reason for the possibility of misdiagnosis. Analyzing this information can be used to extract issues for improving diagnosis quality and provide feedback to physicians.

The alert methods exemplified above may be used alone or in combination. The selection of alert methods can be flexibly made according to the policy of the medical institution, the characteristics of the department, the proficiency of individual physicians, etc.

This disclosure also includes the following configurations:

an answer input unit configured to accept an answer from a patient to a question for differentiating a mental disorder; a video image acquisition unit configured to acquire a video image capturing the patient; a biometric reaction detection unit configured to analyze the video image to detect a change in biometric reaction of the patient; and an estimation unit configured to estimate the mental disorder that the patient suffers from by inputting the accepted answer and the detected change in biometric reaction into a learning model, wherein the learning model is trained on the answer, the change in biometric reaction, and the mental disorder. An information processing system comprising:

a diagnosis result input unit configured to accept a diagnosis result of the mental disorder of the patient by a medical professional based on the answer; and a misdiagnosis possibility determination unit configured to determine whether the estimation result of the mental disorder by the estimation unit matches the diagnosis result or not. The information processing system according to Item 1, comprising:

The information processing system according to Item 1, wherein the estimation unit calculates probabilities that the patient suffers from each of a plurality of mental disorders.

The information processing system according to Item 1, wherein the video image acquisition unit acquires the video image capturing the patient engaging in daily conversation.

a daily conversation processing unit configured to conduct daily conversation with the patient and generate second speech content to convey to the patient in response to first speech content from the patient; wherein the video image acquisition unit acquires the video image capturing conversation between the patient and the daily conversation processing unit. The information processing system according to Item 4, comprising:

accepting an answer from a patient to a question for differentiating a mental disorder; acquiring a video image capturing the patient; analyzing the video image to detect a change in biometric reaction of the patient; and estimating the mental disorder that the patient suffers from by inputting the accepted answer and the detected change in biometric reaction into a learning model, wherein the learning model is trained on the answer, the change in biometric reaction, and the mental disorder. An information processing method comprising steps performed by a computer of:

accepting an answer from a patient to a question for differentiating a mental disorder; acquiring a video image capturing the patient; analyzing the video image to detect a change in biometric reaction of the patient; and estimating the mental disorder that the patient suffers from by inputting the accepted answer and the detected change in biometric reaction into a learning model, wherein the learning model is trained on the answer, the change in biometric reaction, and the mental disorder. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

1 physician terminal 2 management server 3 patient terminal