AI in talent assessment can vary widely depending on its use case and where it is implemented in the resume-to-hire and post employment flow.
AI is most commonly used in four areas of the pre-hire recruitment and assessment process:
1. Conversational AI at first stage in text conversation assessment with dynamic support responses
2. Candidate skill matching in job boards and job marketplace applications generating matches based on resume data and job requirements and descriptions
3. Workflow automation automating a reject or pass decision making process based on decisions and results at each screening stage. Eg. First stage resume screening to second stage assessment or Second stage assessment to schedule a live interview
4. Competency-based assessment assesses the results of behavioral interview responses using natural language understanding to identify language and phrases used in responses that are predicted and identified
videoBIO uses the fourth method, evaluating interview responses for competency-based scoring. Employers can ask candidates questions using the video or audio format questionnaire format delivered through the video interviewing platform and receive honest, unscripted responses from candidates. Employers appreciate this interview method for evaluating candidates based on their unprepared, unscripted responses which provides the opportunity for candidates to provide a “like live” interview response in a one-way environment. Candidates also enjoy having the ability to respond in a less restricting or prescriptive environment often citing their preference for Q&A format in the interview process. Using AI assessment in the analysis of these responses helps employers to more efficiently assess these responses when dealing with high volume of responses.
The Machine Training Process for Competency-based Language Assessment
When building an AI model there are many steps involved in how the model is classified, designed, tested, validated and deployed in order to ensure accuracy and bias mitigation.
videoBIO uses AI for linguistic evaluation using natural language understanding. Our models are built within the IBM Watson Knowledge Studio framework and use language-based data. No visual or facial data is used in our dataset. All data generated from video-responses is automatically transcribed into a text-based interview transcript and then run through the algorithm for assessment. videoBIO only uses interview response data to our competency-based interview questionnaires so no personally identifiable information is collected on any candidate including name, gender, race or other demographic data.
The following are the steps involved in the training of a machine algorithm:
1. Defining the type and classification system including language definition, entities or competency factors being assessed and metrics used (i.e. mentions, sentiment, correlation)
2. Defining the model type. We use rules-based systems that are trained in a supervised annotation environment, and then deployed. This prevents generalized, ongoing learning by the algorithm where new data or bias could be introduced without supervision. Once the model is trained, evaluated and deployed, it is deployed.
3. Defining the dataset. This includes defining the type of interview data and how the data is collected. We use only interview question response data with uniformity across the same questions, asked the same way and tailored to each individual competency assessment. We do not use generalized or universal data in our training environment which allows us to have highly precise and high performing algorithms and F1 scores.
4. Machine training involves multiple stages of human annotator training, annotation, supervision across thousands of interview transcript documents. This is a progressive process which involves multiple stages of training, deploying, evaluation and review, retraining, retesting and supervision. This is a continuous process until we have achieved our target F1 score indicating a high performing algorithm.
5. Testing and evaluation. Throughout the process, each set is tested and evaluated in small sets of 250-500 annotated documents. After the sets have been annotated by at least two annotators and then supervised and checked, they are run and assessed for machine performance which looks at precision and recall based on the entities and competencies being predicted. If there are areas where precision or recall are lower than threshold, the annotators will concentrate on these areas to improve the overall accuracy of predictions by reducing machine confusion by adding additional data and updating annotations.
6.
Validation of results. We validate our results in a variety of methods which include cross correlation with other test types including multiple choice and traditional assessment scores. For more information on validity visit
An Introduction to AI and its use in Talent Measurement
To learn more about videoBIO’s pre-trained AI assessments or custom trained AI environments for companies who are interested in training based on their own interview data, please
contact us for a demonstration.
