System and Method for Generation and Use of Data Models for Determination of Inferred Skills

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

Embodiments described herein are generally directed to computer-based data analytics and the use of data models for determining inferred characteristics associated with candidates, for example inferred skills, for display within or as part of analytics dashboards, scorecards, or other data visualizations.

Generally described, within an enterprise organization, data analytics enables the computer-based examination of amounts of data, to derive conclusions or other information from the data.

For example, business intelligence software tools can be used to provide the organization with information describing their enterprise data, in a format that enables users to make strategic business decisions.

Examples of the types of data analytics of interest to organizations include those directed to Enterprise Resource Planning (ERP), Human Capital Management (HCM), or Human Resources (HR), which can be used to address various use cases. For example, an HCM/HR system can be used within an organization to maintain a database of information descriptive of the organization's employees and their particular work-related skills.

However, in some organizations, the information associated with each employee in an HCM/HR system or database, for example as part of the employee's talent profile, may not be updated on a sufficiently-regular basis to accurately describe ongoing updates to the employee's known or potential skill-set. This makes it challenging for an organization's HCM/HR department or its business leaders to properly assess their workforce's overall skills, or to identify potential skill gaps that may need to be addressed.

Embodiments described herein are generally directed to computer-based data analytics and the processing of enterprise data, including the generation and use of data models for determining inferred characteristics associated with candidates.

In accordance with an embodiment, data-processing pipelines comprising artificial intelligence or machine learning (AI/ML) data models are used to process structured, semi-structured, and/or unstructured sets of data, received from various sources; generate a multi-dimensional ontology and a taxonomy associated with the characteristics of open positions or potential candidates; identify, based on the data models, one or more additional or inferred characteristics associated with the candidates; and present the output by way of an analytics dashboard, scorecard, or other data visualization.

For example, in accordance with an embodiment, the described approach can be used to identify a candidate's otherwise-hidden or inferred skills, which in turn provides an organization's HCM/HR department and business leaders with a more complete understanding of their workforce's overall capabilities. Data visualizations can be used, for example, to identify employee skills suitable for planned projects or business goals, undertake initiatives to address potential skill gaps, or support particular groups of employees in their ongoing skill development.

Generally described, within an enterprise organization, data analytics enables the computer-based examination of amounts of data, to derive conclusions or other information from the data.

For example, business intelligence software tools can be used to provide the organization with information describing their enterprise data, in a format that enables users to make strategic business decisions.

Examples of the types of data analytics of interest to organizations include those directed to Enterprise Resource Planning (ERP), Human Capital Management (HCM), or Human Resources (HR), which can be used to address various use cases. For example, an HCM/HR system can be used within an organization to maintain a database of information descriptive of the organization's employees and their particular work-related skills.

Increasingly, a data analytics environment can be provided within the context of software-as-a-service (SaaS) or cloud-based enterprise software environments, such as, for example, an Oracle Fusion Applications, Oracle Analytics Cloud or Fusion Analytics Warehouse (FAW) environment.

1 FIG. illustrates an example data analytics environment, in accordance with an embodiment.

1 FIG. 1 FIG. The embodiment illustrated inis provided for illustrating an example data analytics environment in association with which various embodiments described herein can be used. The components and processes illustrated inand as described elsewhere herein with regard to various other embodiments, can be provided as software or program code executable by, for example, a cloud computing system, or other suitably-programmed computer system.

1 FIG. 100 101 102 104 160 161 As illustrated in, in accordance with an embodiment, a data analytics environmentcan be provided by, or otherwise operate at, a computer system having a computer hardware (e.g., processor, memory), and including one or more software components operating as a control plane, and a data plane, and providing access to a data warehouse instance(e.g., having a database, or other type of data source).

110 111 In accordance with an embodiment, the control plane operates to provide control for cloud or other software products offered within the context of a cloud environment. For example, in accordance with an embodiment, the control plane can include a console interfacethat enables access by a customer (tenant) and/or a cloud environment having a provisioning component, for example to allow customers to provision services for use within their enterprise environment. The provisioning component can provision a data warehouse instance, including a customer schema of the data warehouse; and populate the data warehouse instance with the appropriate information supplied by the customer.

120 134 In accordance with an embodiment, the data plane can include a data pipeline or process layerand a data transformation layer, that together process data from an organization's enterprise software environment, and load a transformed data into the data warehouse. The data transformation layer can include a data model, such as, for example, a knowledge model (KM), or other type of data model, that the system uses to transform the data received from business applications and corresponding databases, into a model format understood by the data analytics environment. The data plane is responsible for performing extract, transform, and load (ETL) operations, including extracting data from an organization's enterprise software environment, transforming the extracted data into a model format, and loading the transformed data into a customer schema of the data warehouse.

106 For example, in accordance with an embodiment, each customer (tenant) of the environment can be associated with their own customer schema; and can be additionally provided with read-only access to the data analytics schema, which can be updated by a data pipeline or process, for example, an ETL process, on a periodic or other basis. For example, a data pipeline or process can be scheduled to execute at intervals (e.g., hourly/daily/weekly) to extract data from an enterprise software environment, such as, for example, business productivity software applications and corresponding databases.

108 In accordance with an embodiment, an extract processcan extract the data, whereupon extraction the data pipeline or process can insert extracted data into a data staging area, which can act as a temporary staging area for the extracted data. When the extract process has completed its extraction, the data transformation layer can be used to transform the extracted data into a model format to be loaded into the customer schema of the data warehouse. During the data transformation, the system can perform dimension generation, fact generation, and aggregate generation, as appropriate. Dimension generation can include generating dimensions or fields for loading into the data warehouse instance.

150 In accordance with an embodiment, after transformation of the extracted data, the data pipeline or process can execute a warehouse load procedure, to load the transformed data into the customer schema of the data warehouse instance. Subsequent to the loading of the transformed data into customer schema, the transformed data can be analyzed and used in a variety of additional business intelligence processes.

180 190 Different customers may have different requirements with regard to how their data is classified, aggregated, or transformed, for providing data analytics or business intelligence data, or developing software analytic applications. In accordance with an embodiment, to support such different requirements, a semantic layercan include data defining a semantic model of a customer's data; which is useful in assisting users in understanding and accessing that data using commonly-understood business terms; and provide custom content to a presentation layer.

In accordance with an embodiment, a customer may perform modifications to their data source model, to support their particular requirements, for example by adding custom facts or dimensions associated with the data stored in their data warehouse instance; and the system can extend the semantic model accordingly. A semantic model can be defined, for example, in an Oracle environment, as a BI Repository (RPD) file, having metadata that defines logical schemas, physical schemas, physical-to-logical mappings, aggregate table navigation, and/or other constructs that implement the various physical layer, business model and mapping layer, and presentation layer aspects of the semantic model.

In accordance with an embodiment, the presentation layer can enable access to the data content using, for example, a software analytic application, user interface, analytics dashboard, key performance indicators (KPI's); or other type of report or interface as may be provided by products such as, for example, Oracle Analytics Cloud, or Oracle Analytics for Applications.

18 56 In accordance with an embodiment, a query engine(e.g., an Oracle Business Intelligence Server, OBIS instance) operates in the manner of a federated query engine to serve analytical queries or requests from clients directed to data stored at a database. The query engine can push down operations to supported databases, in accordance with a query execution plan, wherein a logical query can include Structured Query Language (SQL) statements received from the clients; while a physical query includes database-specific statements that the query engine sends to the database to retrieve data when processing the logical query.

10 11 12 14 In accordance with an embodiment, a user/developer can interact with a client computer devicethat includes a computer hardware(e.g., processor, storage, memory), user interface, and client application. A query engine or business intelligence server generally operates to process inbound, e.g., SQL, requests against a database model, build and execute one or more physical database queries, process the data appropriately, and return the data in response to the request.

To accomplish this, in accordance with an embodiment, the query engine can include a logical or business model, or metadata, that describes the data available as subject areas for queries; a request generator that takes incoming queries and turns them into physical queries for use with a connected data source; and a navigator that takes the incoming query, navigates the logical model and generates those physical queries that best return the data required for a particular query.

For example, in accordance with an embodiment, the query engine may employ a logical model mapped to data in a data warehouse, by creating a simplified star schema business model over various data sources so that the user can query data as if it originated at a single source. The information can then be returned to the presentation layer as subject areas, according to business model layer mapping rules.

In accordance with an embodiment, the query engine can process queries against a database according to a query execution plan. During operation the query engine can create a query execution plan which can then be further optimized, for example to perform aggregations of data necessary to respond to a request. Data can be combined together and further calculations applied, before the results are returned to the calling application.

196 In accordance with an embodiment, a request for data analytics or visualization information can be received via a client application and user interface as described above, and communicated to the data analytics environment (in the example of a cloud environment, via a cloud service). The system can retrieve an appropriate dataset to address the user/business context, for use in generating and returning the requested data analytics or visualization information to the client, as a data visualization.

In accordance with an embodiment, a client application can be implemented as software or computer-readable program code executable by a computer system or processing device, and having a user interface, such as, for example, a software application user interface or a web browser interface. The client application can retrieve or access data via an Internet/HTTP or other type of network connection to the data analytics environment, or in the example of a cloud environment via a cloud service provided by the environment.

2 FIG. further illustrates an example data analytics environment, in accordance with an embodiment.

2 FIG. 198 As illustrated in, in accordance with an embodiment, the data analytics environment enables a dataset to be retrieved, received, or prepared from one or more data source(s), for example via one or more data source connections. Examples of the types of data that can be transformed, analyzed, or visualized using the systems and methods described herein include data directed to Enterprise Resource Planning (ERP), Human Capital Management (HCM), or Human Resources (HR), or other types of data provided at one or more of a database, data storage service, or other type of data repository or data source.

For example, in accordance with an embodiment, a request for data analytics or visualization information can be received via a client application and user interface as described above, and communicated to the data analytics environment, for example via a cloud service. The system can retrieve an appropriate dataset to address the user/business context, for use in generating and returning the requested data analytics or visualization information to the client.

3 FIG. further illustrates an example data analytics environment, in accordance with an embodiment.

3 FIG. 106 109 107 105 As illustrated in, in accordance with an embodiment, data can be sourced, e.g., from a customer's (tenant's) enterprise software environment (), using the data pipeline process; or as custom datasourced from one or more customer-specific applications; and loaded to a data warehouse instance, including in some examples the use of an object storagefor storage of the data. A user can create a dataset that uses tables from different connections and schemas. The system uses the relationships defined between these tables to create relationships or joins in the dataset.

162 164 114 117 In accordance with an embodiment, the data warehouse can include a default data analytics schemaand, for each customer (tenant) of the system, a customer schema. For each customer (tenant), the system uses the data analytics schema that is maintained and updated by the system, within a system/cloud tenancy, to pre-populate a data warehouse instance for the customer, based on an analysis of the data within that customer's enterprise applications environment, and within a customer tenancy. As such, the data analytics schema maintained by the system enables data to be retrieved, by the data pipeline or process, from the customer's environment, and loaded to the customer's data warehouse instance.

In accordance with an embodiment, the system also provides, for each customer of the environment, a customer schema that allows the customer to supplement and utilize the data within their own data warehouse instance. For each customer, their resultant data warehouse instance operates as a database whose contents are partly-controlled by the customer; and partly-controlled by the environment (system).

For example, in accordance with an embodiment, a data warehouse can include a data analytics schema and, for each customer/tenant, a customer schema sourced from their enterprise software environment. The data provisioned in a data warehouse tenancy is accessible only to that tenant; while at the same time allowing access to various, e.g., ETL-related or other features of the shared environment.

In accordance with an embodiment, for a particular customer/tenant, upon extraction of their data, the data pipeline or process can insert the extracted data into a data staging area for the tenant, which can act as a temporary staging area for the extracted data. When the extract process has completed its extraction, the data transformation layer can be used to transform the extracted data into a model format to be loaded into the customer schema of the data warehouse.

4 FIG. further illustrates an example data analytics environment, in accordance with an embodiment.

4 FIG. 160 163 165 167 170 As illustrated in, in accordance with an embodiment, the process of extracting data from a customer's (tenant's) enterprise software environment, and loading the data to a data warehouse instance, or refreshing the data in a data warehouse, generally involves several stages, performed by an ETP serviceor process, including one or more extraction service; transformation service; and load/publish service, executed by one or more compute instance(s).

For example, in accordance with an embodiment, extracted files can be uploaded to an object storage component for storage of the data. The transformation process then applies a business logic while loading them to a target data warehouse, e.g., an Autonomous Data Warehouse (ADW) database, which is internal to the data pipeline or process, and is not exposed to the customer (tenant). A load/publish service or process takes the data from the ADW database and publishes it to a data warehouse instance that is accessible to the customer (tenant).

5 FIG. further illustrates an example data analytics environment, in accordance with an embodiment.

5 FIG. 180 182 162 162 106 106 181 183 160 160 As illustrated in, in accordance with an embodiment, the data pipeline or process maintains, for each of a plurality of customers (tenants), for example customer A, customer B, a data analytics schema that is updated on a periodic basis, by the system in accordance with best practices for a particular analytics use case. For each of a plurality of customers (e.g., customers A, B), the system uses the data analytics schemaA,B, that is maintained and updated by the system, to pre-populate a data warehouse instance for the customer, based on an analysis of the data within that customer's enterprise applications environmentA,B, and within each customer's tenancy (e.g., customer A tenancy, customer B tenancy); so that data is retrieved, by the data pipeline or process, from the customer's environment, and loaded to the customer's data warehouse instanceA,B.

164 164 In accordance with an embodiment, the data analytics environment also provides, for each of a plurality of customers of the environment, a customer schema (e.g., customer A schemaA, customer B schemaB) that allows the customer to supplement and utilize the data within their own data warehouse instance.

108 108 As described above, in accordance with an embodiment, for each of a plurality of customers of the data analytics environment, their resultant data warehouse instance operates as a database whose contents are partly-controlled by the customer; and partly-controlled by the data analytics environment (system); including that their database appears pre-populated with appropriate data that has been retrieved from their enterprise applications environment to address various analytics use cases. When the extract processA,B for a particular customer has completed its extraction, the data transformation layer can be used to transform the extracted data into a model format to be loaded into the customer schema of the data warehouse.

186 In accordance with an embodiment, activation planscan be used to control the operation of the data pipeline or process services for a customer, for a particular functional area, to address that customer's (tenant's) particular needs. For example, an activation plan can define a number of extract, transform, and load (publish) services or steps to be run in a certain order, at a certain time of day, and within a certain window of time.

6 FIG. further illustrates an example data analytics environment, in accordance with an embodiment.

Generally described, within a database or data warehouse, the data of interest may be spread across multiple tables. In such environments, joins can be used to stitch the data from various tables together, to better prepare the data for analysis.

6 FIG. 210 216 221 227 302 304 For example, as illustrated in, in accordance with an embodiment, the data analytics environment enables a dataset to be retrieved, received, or prepared from one or more data source(s), for example via one or more data source connections, fact and/or dimension tables-, or joins-between selections of dimension tables,.

192 232 In accordance with an embodiment, a request received at a data visualization environment to display analytic artifacts, for example as may be related to key performance indicators, analytics dashboards, or scorecards, can be received via a client application and user interface as described above, and communicated to the data analytics environment via a cloud service. The system can retrievean appropriate dataset using, e.g., SELECT statements, to address the user/business context, for use in generating and returning the requested data analytics or visualization information to the client.

As described above, business intelligence software tools can be used to provide an organization with information describing their enterprise data, in a format that enables users to make strategic business decisions; with examples of the types of data analytics of interest including those directed to Enterprise Resource Planning (ERP), Human Capital Management (HCM), or Human Resources (HR)

In some environments, a variety of key measures or metrics are used to assess, quantify, or provide an indication of the effectiveness of an organization group. For example, a team effectiveness KPI can be used to measure effectiveness of a particular team by assessing its attrition rate, cost, requisition filling rate, or engagement survey score etc., of the team in comparison with the rest of the organization. Such key measures help executives to identify specific managers with high performing teams, and/or assess potential areas of improvement.

7 FIG. illustrates an example use of a data analytics environment to provide a visualization of key performance indicators or scorecards, in accordance with an embodiment.

7 FIG. 280 270 290 As illustrated in, in accordance with an embodiment, a data analytics environment, for example as described above, can include a KPI generatorthat receives information via a data layerfrom a data warehouse instance, and enables generation of enterprise data (e.g., HCM/HR) KPIsas analytics dashboards, scorecard, or other data visualizations associated with the enterprise organization.

282 284 300 310 In accordance with an embodiment, the system can be accessed by a user using a client computer device, for example as described above. In response to a request, the system can receive, from a data warehouse instance, an (e.g., HCM/HR) enterprise data, and generate a user interface, analytics dashboard, or KPI, for use as one or more data visualizations, and for subsequent display at a client user interface, for example as a two-dimensional analytics dashboard, scorecard, or other data visualization format.

311 312 319 For example, when used with HCM/HR enterprise data, such data can include for example, a recruiting cloud data, HR core data, or additional HR data, as further described below.

In accordance with various embodiments, the teachings described herein can be used with various systems and methods for providing or supporting the use of data analytics or KPIs, such as, for example, the systems and methods described in U.S. Patent Application titled “SYSTEM AND METHOD FOR DATA ANALYTICS WITH AN ANALYTIC APPLICATIONS ENVIRONMENT”, application Ser. No. 16/862,394, filed Apr. 29, 2020, and subsequently published as U.S. Patent Application Publication No. 2020/0349155 on Nov. 5, 2020; or U.S. Patent Application titled “TECHNIQUES FOR DATA-DRIVEN CORRELATION OF METRICS”, application Ser. No. 16/586,347, filed Sep. 27, 2019, and subsequently published as U.S. Patent Application Publication No. 2020/0104775 on Apr. 2, 2020; each of which patent applications and the contents thereof are herein incorporated by reference.

As described above by way of example, an HCM/HR system can be used within an organization to maintain a database of information descriptive of the organization's employees and their particular work-related skills.

However, in some organizations, the information associated with each employee in an HCM/HR system or database, for example as part of the employee's talent profile, may not be updated on a sufficiently-regular basis to accurately describe ongoing updates to the employee's known or potential skill-set. This makes it challenging for an organization's HCM/HR department or its business leaders to properly assess their workforce's overall skills, or to identify potential skill gaps that may need to be addressed.

For example, although a HCM system may offer a means by which employees can describe their current skills, which forms part of the employee's talent profile-this generally requires the employees to be proactive in updating their talent profiles as they take on new types of projects, receive further training, or learn new skills.

Within a particular industry, traditional job titles may evolve to encompass new duties; and employees with the same job title might have expanded their skills in different directions beyond those initially required of the position.

Different job titles may also be used for otherwise similar positions, such that relying on the use of job titles may not provide a clear indication of the underlying responsibilities.

For open positions, there be a large number of job applicants or candidates, and with the variety of different ways in which candidates might describe their former jobs and skills, finding the most appropriate candidate can be time-consuming.

To address the above aspects, in accordance with an embodiment, data-processing pipelines comprising artificial intelligence or machine learning (AI/ML) data models are used to process structured, semi-structured, and/or unstructured sets of data, received from various sources; generate a multi-dimensional ontology and a taxonomy associated with the characteristics of open positions or potential candidates; identify, based on the data models, one or more additional or inferred characteristics associated with the candidates; and present the output by way of an analytics dashboard, scorecard, or other data visualization.

For example, in accordance with an embodiment, the described approach can be used to identify a candidate's otherwise-hidden or inferred skills, which in turn provides an organization's HCM/HR department and business leaders with a more complete understanding of their workforce's overall capabilities. Data visualizations can be used, for example, to identify employee skills suitable for planned projects or business goals, undertake initiatives to address potential skill gaps, or support particular groups of employees in their ongoing skill development.

In accordance with an embodiment, the system can identify, enrich, and augment an identification of existing skills based on data received from diverse sources, automatically uncovering otherwise-hidden skills for the benefit of individual employees and the organization as a whole. For example, as further described herein, the system can be used to determine any combination of one or more:

Existing skills, which are generally those skills associated with a candidate and which may be found on the candidate's talent profile.

Inferred skills, which are generally skills that may not be found on the candidate's talent profile; however the system indicates, based on a particular (e.g., data analytics, HCM/HR) customer's data and requirements, that such skills should be included on their talent profile.

Suggested skills, which are generally skills that likewise may not be found on the candidate's talent profile, however the system indicates, based on based on a particular (e.g., data analytics, HCM/HR) customer's data and requirements, along with additional external data such as that provided by a recruiting environment or by external job applicants or candidates, that such skills should be included on their talent profile.

8 FIG. illustrates a system for generation and use of data models for determination of inferred characteristics, in accordance with an embodiment.

Generally described, the determination of a candidate's inferred characteristics (for example, their inferred skills) is a two-step process, including:

In a first step: extracting an initial skills-related data from one or more provide text documents, using AI/ML data models; and

In a second step: after extracting the skills-related data, using statistical techniques to predict inferred skills for particular candidates.

8 FIG. 402 406 408 As illustrated in, In accordance with an embodiment, the system comprises a dynamic skills pipelinethat includes a plurality of dynamic skills runtime recommendation models, that operate together to process an ingested data; and a dynamic skills APIby which an indication of inferred characteristics, for example inferred skills, can be surfaced to other components, for example to generate an analytics dashboard, scorecard, or other data visualization.

412 416 In accordance with an embodiment, to support the determination of inferred skills, the system further comprises a skills ontology, providing a framework for assessing job-titles-to-skills relationships; and a skills taxonomy, providing a framework for assessing skills-to-skills relationships.

420 432 434 436 430 In accordance with an embodiment, the dynamic skills pipeline operates on an ingested data, which can include both structured data such as HCM/HR enterprise data, recruiting cloud data, HR core data, or additional HR data, as described above; and can also include various or additional structured data, semi-structured data, and/or un-structured datareceived from various sources as additional/external candidate data.

9 FIG. illustrates how the system can assess structured, semi-structured, and/or unstructured data, in accordance with an embodiment.

9 FIG. 444 446 As illustrated in, in accordance with an embodiment, additional examples of structured data that may be relevant to the determination of a candidate's inferred skills can include HCM/HR enterprise data such as HCM core data, HCM learning data, or additional HCM/HR datasets.

442 In accordance with an embodiment, examples of semi-structured data that may be relevant to the determination of a candidate's inferred skills can include their skills dataas populated, for example, by their submitted resume or a talent profile.

In accordance with an embodiment, examples of un-structured data that may be relevant to the determination of a candidate's inferred skills can include, for example, their social media profiles, or response to surveys.

10 FIG. illustrates an overview of how the system uses AI/ML data models to determine inferred characteristics, in accordance with an embodiment.

10 FIG. 451 As illustrated in, in accordance with an embodiment, during a first phase () of the process, direct data collection campaigns can be used, for example, to encourage employee skill entry.

452 During a second phase () of the process, an initial reporting and analysis can be used to determine those reports that will be used to summarize data entered by employees.

452 During a third phase () of the process, the system can supplement employee-entered data using AI/ML data models.

451 During a fourth phase () of the process, using skill gap analytics or similar techniques, the system can provide data analytics information using employee entered and inferred skills.

11 FIG. further illustrates a system for generation and use of data models for determination of inferred characteristics, in accordance with an embodiment.

11 FIG. As illustrated in, in accordance with an embodiment, the dynamic skills pipeline includes a plurality of pipelines and data models that operate on ingested data, and provide an indication of extracted skills, suggested skills, and/or inferred skills to the dynamic skills API, for surfacing and use by other components, for example to generate an analytics dashboard, scorecard, or other data visualization.

460 462 464 466 467 468 470 In accordance with an embodiment, the pipelines can include, for example an extraction pipeline, comprising a skill predictor, named entity recognition (NER) Model, relevancy classifier model, and industry classifier model, that are used by the system determine a corpus of predicted skills, and thereafter provide to the dynamic skills API an extracted skills dataassociated with one or more candidates.

480 482 484 486 490 In accordance with an embodiment, an inventory pipelinecomprising a skill normalize model, and job title normalize model, can be used to determine a normalized title-to-skill map, and thereafter provide to the dynamic skills API a suggested skills dataassociated with the one or more candidates.

500 502 504 510 In accordance with an embodiment, an inferred skill pipelinecomprising an inferred skill modelcan be used to determine an employee's inferred skills, and thereafter provide to the dynamic skills API an inferred skills dataassociated with the one or more candidates.

520 522 526 In accordance with an embodiment, additional pipelines, such as a confidence score pipelinecomprising a confidence modelcan be used to determine confidence scoresassociated with the normalized title-to-skill map ore other aspects of the process.

In accordance with an embodiment, the pipelines of data models can be used to create, populate, and update the skills ontology and skills taxonomy.

In accordance with an embodiment, the system can then operate, for example on a position requisition dataset that indicates one or more skills of interest; and a talent profiles dataset that indicates skills associated with one or more candidates'talent profiles. The skills ontology and skills taxonomy can be overlaid onto the (position requisition and talent profiles) datasets, to determine with a measure of confidence which particular candidates may have likely or inferred skills matching the skills of interest associated with a particular job requisition.

In accordance with an embodiment, for a particular candidate, based on the examination of such data, the system can determine one or more suggested or inferred skills for the candidate, and recommend that the skills be added to their talent profile, for present or future use.

In accordance with an embodiment, during development of the system for a particular organization, the dynamic skills pipeline can be initially trained on an organization's existing databases of job requisitions, to provide an initial skills ontology, skills taxonomy, and data models that are well-suited to that organization's particular environment, The data models can then be updated on a regular basis from that point forward. Generally the system is operated so as to determine inferred skills on a batch basis, for many or all of an organization's employees at a time, providing an overall view into any skill gaps within the organization.

In accordance with an embodiment, the data models used to extract skills from an ingested data, for example a given text document, can include:

In accordance with an embodiment, the named entity recognition (NER) model is a deep learning language contextual model; the purpose of which is to extract seen and unseen skills from a given text document.

In accordance with an embodiment, the data model can be trained on historical job requisitions, which can be manually-tagged with a set of known skills. The data model is then trained to learn the grammatical usage of skills in a text document. Once trained, the data model is capable of identifying “new and unseen skills” from any given text document.

In accordance with an embodiment, the input to the data model is a text in paragraph form; and the model output is list of skills mentioned in the paragraph.

In accordance with an embodiment, the relevancy classifier model is a binary classifier, the purpose of which is to predict the correctness of skills extracted by the NER model.

In accordance with an embodiment, the data model can be trained on manually-labeled positive and negative samples provided by NER predictions.

In accordance with an embodiment, the input to the data model is a pair of sentence and skill in the sentence as predicted by the NER model; and the model output is an indication of true/false.

In accordance with an embodiment, the skill normalizer model is a language grammatical model, the purpose of which is to identify the best form of a skills given millions of skills.

In accordance with an embodiment, the data model can be trained using a two-step approach:

In a first step, the system groups skills syntactically and semantically. Syntactical grouping can be performed according to grammatical rules; and semantic grouping can be performed by an in-house trained embedding (e.g., doc2vec) model. For example, [managed people and people management] skills can be grouped syntactically using grammatical rules; whereas [margin lending and margin financing] skills can be grouped semantically using vector embedding models.

In a second step, once the group is formed, the system operates to predict the best form of skill within the group; and the remaining skills in that group become variants. This can be accomplished by observing the frequencies at which these skills appear in a data corpus overlaid by an un-contextual grammar model. For example, in an example group [MS Excel, Microsoft Excel, Excel 365], the data model may elect to treat [Microsoft Excel] as the normalized form and the remainder of the terms as variants.

In accordance with an embodiment, the input to the data model input is a corpus (a large amount, e.g., a million) skills; and the model output is the normalized forms of those skills and their variants.

In accordance with an embodiment, the job title normalizer model is a vector similarity-based model, the purpose of which is to convert a given job title to its normalized form, removing noise and identifying seniority. The normalized forms of titles can be based on an industry-recognized nomenclature of occupations such as that defined by the European Skills, Competences, Qualifications and Occupations (ESCO) or the U.S. Occupational Information Network (O*NET) programs.

In accordance with an embodiment, the data model can be trained using a multi-step approach:

Remove noise: in this step, the NER model is trained to extract person/place/time entities which are considered noise; for example in [software developer-Austin], the reference to Austin in this example may not be needed.

Identifying seniority: a fuzzy match seniority of keywords is performed using a pre-curated list of seniority keywords; for example in [sr. software developer, senior software developer], the keywords sr. and senior in this example are extracted and added at the end.

Semantic similarity: a semantic similarity comparison with, e.g., the ESCO/O*NET job titles, is performed by an in-house trained embedding (e.g., doc2vec) model.

In accordance with an embodiment, the input to the model is a job title; and the model output is the normalized forms of the job title.

12 FIG. further illustrates a system for generation and use of data models for determination of inferred characteristics, in accordance with an embodiment.

12 FIG. 467 485 540 544 548 As illustrated in, in accordance with an embodiment, the system can include additional models as needed to address particular use cases, such as, for example, an industry classifier model, mapping aggregator, and/or additional modelsand skill-related maps, that can be used to provide additional skill-related output.

The above example of data models and their operation, including the training of the various data models, model inputs, and model outputs, are provided by way of example. In accordance with various embodiments, additional types of data models can be used to address particular use cases.

13 18 FIGS.- illustrate how the system can generate data visualizations of inferred characteristics, for example, inferred skills associated with candidates, in accordance with an embodiment.

13 FIG. As illustrated in, in accordance with an embodiment, based on certain conditions the system can indicate within the user interface an analytics dashboard, scorecard, or other data visualization of measures with values or colors. Optionally, the system can display observations or insights related to the data; or can allow a user to drill-down or otherwise interact with the visualization.

14 FIG. As illustrated in, in accordance with an embodiment, areas within the data visualization indicative of particular inferred characteristics or skills can be sized, or arranged within the data visualization to illustrate their value or magnitude compared with other inferred characteristics or skills.

15 FIG. As illustrated in, in accordance with an embodiment, areas within the data visualization indicative of particular inferred characteristics or skills can additionally be shaded or colored within the data visualization to provide contrast, or to illustrate their value or magnitude compared with other inferred characteristics or skills.

16 FIG. As illustrated in, in accordance with an embodiment, areas within the data visualization indicative of particular inferred characteristics or skills can additionally be labeled with the corresponding characteristic or skill.

17 FIG. As illustrated in, in accordance with an embodiment, the user interface or analytics dashboard can include multiple data visualizations with areas displayed therein indicative of, in this example, existing skills, inferred skills, and suggested skills.

18 FIG. As illustrated in, in accordance with an embodiment, the user interface or analytics dashboard can include different types of data visualizations, in this example a radar chart, with areas displayed therein similarly indicative of, in this example, existing skills, inferred skills, and suggested skills.

The above examples of analytics dashboards, scorecards, or other data visualizations are provided by way of example for purposes of illustrating the various techniques described herein. In accordance with various embodiments, the system can include other types of analytics dashboards, scorecards, or data visualizations useful for indicating and reporting inferred characteristics or skills.

19 FIG. illustrates an example process or method for generation and use of data models for determination of inferred characteristics associated with candidates, in accordance with an embodiment.

19 FIG. 562 As illustrated in, in accordance with an embodiment, at step, the process includes providing, at a computer system having a computer hardware, a data analytics environment that includes or provides access to a data warehouse instance for storage of enterprise data.

564 At step, the system can retrieve into the data analytics environment a structured, semi-structured, and/or unstructured data, received from various sources.

566 At step, the system can generate a multi-dimensional ontology and a taxonomy associated with the characteristics of open positions or potential candidates, for use in processing the received data.

568 At step, the system can then identify from the processed data one or more additional or inferred characteristics associated with a candidate, and output data indicative of the candidate's inferred characteristics, which can then be displayed by way of an analytics dashboard, scorecard, or other data visualization.

For example, in accordance with an embodiment, the described approach can be used to identify a candidate's otherwise-hidden or inferred skills, which in turn provides an organization's HCM/HR department and business leaders with a more complete understanding of their workforce's overall capabilities. For example analytics dashboards, scorecards, or data visualizations as described herein can be used to identify employee skills suitable for planned projects or business goals, undertake initiatives to address potential skill gaps, or support particular groups of employees in their ongoing skill development.

20 25 Figures- Illustrate an Example Use Case of Determining Inferred Skills, in accordance with an embodiment.

20 25 FIGS.- In particular,illustrate an example use case in which a HR professional is tasked with assessing the possibility of onboarding candidates for a software development project due to begin in the coming weeks, and which is expected to require software engineers proficient in Python.

20 FIG. As illustrated in, in accordance with an embodiment, the HR professional can display a talent analytics dashboard that provides at a high level various KPIs of interest, including in this example, open requisitions, job openings by role, and candidates by source and role.

21 FIG. As illustrated in, in accordance with an embodiment, the talent analytics dashboard reveals that the engineering department has the second highest number of open requisitions, with the highest number of job applications being software engineers. Based on this it appears hiring external candidates with python skills may be challenging.

22 FIG. As illustrated in, in accordance with an embodiment, considering internal candidates instead within the organization, the HR professional generates an existing skills analytics dashboard, illustrating the distribution of employee skills across the entire organization, drawn from the employee talent profiles.

23 FIG. As illustrated in, in accordance with an embodiment, the system can be used to generate inferred and suggested skill sets, using the above-described approach, based on employee job and talent profiles, and ingested additional data where available. Typically these inferred and suggested skills are not visible on the employee's talent profile, but the system suggests they should be included.

24 FIG. As illustrated in, in accordance with an embodiment, since the HR professional is particularly interested in candidates with Python skills, they can use an interface provided by the system that allows filtering or drilling down within this skill set, in this example to see those candidates for whom the system infers as having Python skills.

25 FIG. As illustrated in, in accordance with an embodiment, the system can generate an analytics dashboard, scorecard, or other data visualization in which several candidates with Python skills have been determined, either as a known or inferred skill, from which the HR professional can identify and potentially select for use in the upcoming software development project.

In accordance with an embodiment, examples of various technical advantages or use cases supported by the system include:

Maximizing employee potential with better skills utilization: In determining inferred skills, the described approach enables employees to discover new skills to grow their careers. Business managers are better equipped to assess individual and team readiness for planned projects, connect skilled individuals with the right opportunities, and leverage internal talent to fulfill urgent business needs.

Driving strategic workforce planning with better skills-to-title mapping: The described approach supports the application of skills to varying business requirements; and provides skills-to-titles alignment, for example, to narrow a pool of job applicants or candidates for specific positions.

Adapting to industry trends with agile upskilling/reskilling: Business leaders can track and analyze employee skills to address skill gaps; gain insights into skill penetration at every level from the entire organization to specific employees; and make insight-driven decisions regarding the skills they need to develop or invest in.

The above examples of advantages and use cases are provided by way of example for purposes of illustrating the various techniques described herein. It will be evident that in accordance with various embodiments, additional advantages and use cases can be provided, including those generally directed to workforce and individual employee growth, or to address the changing needs of an enterprise organization or business environment.

In accordance with various embodiments, the teachings herein can be implemented using one or more computer, computing device, machine, or microprocessor, including one or more processors, memory and/or computer readable storage media programmed according to the teachings herein. Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art.

In some embodiments, the teachings herein can include a computer program product which is a non-transitory computer readable storage medium (media) having instructions stored thereon/in which can be used to program a computer to perform any of the processes of the present teachings. Examples of such storage mediums can include, but are not limited to, hard disk drives, hard disks, hard drives, fixed disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, or other types of storage media or devices suitable for non-transitory storage of instructions and/or data.

The foregoing description has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the scope of protection to the precise forms disclosed. Further modifications and variations will be apparent to the practitioner skilled in the art.

The embodiments were chosen and described in order to best explain the principles of the teachings herein and their practical application, thereby enabling others skilled in the art to understand the various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope be defined by the following claims and their equivalents.