In this session we will explore the following:

1. The relationship between reliability and validity in psychometric assessment
2. How psychometric test administrators can impact the reliability of tests

Psychometric Test Reliability

When choosing a reputable test, whether it be aptitude or personality, one of the properties of the test you will need to look for is reliability. We’ll consider reliability in appropriate detail in a later section of the course.  For now, think of reliability as consistency.  In order to have absolute confidence in our test scores we need them to be consistent.  However, we can’t test and retest our candidates in the real world. Despite this, reputable test publishers would already have done this for you. This would have been carried out under optimal conditions.  So, now you know that you are using a reliable test (one that produces consistent scores), it’s your task as the test administrator to ensure that the test remains a reliable test.

Why is reliability so important?

Whenever you assess something, you expect the score you get to be reliable. For example, if you assess your weight using bathroom scales, you expect the reading you get to be consistent across at least the short term. If you weigh yourself over 2 consecutive days and get significantly different readings you know something is wrong with the scales!  The same is true of psychometric tests. The publisher first ensures that the test scores will be consistent over time and then you, as the administrator, need to ensure that your actions do not make the test less reliable.

Not only do we want and expect test results to remain reliable over time, but we also know that reliability is a precursor to validity. It sets an upper limit on the test’s validity. In other words, if your test is not reliable then it is not valid. Confusing?  Let’s use the weighing scales example again…

Let’s suppose a medical doctor does some research which shows that those who weight more than 120kg are significantly more likely to suffer a heart attack.  His research shows that weight is a valid indicator for predicting the heart attack.  The scales are fit for the purpose of predicting a heart attack.  Validity is all about being fit for purpose.  Now if those scales are not reliable, they will provide inconsistent data over the time of the research program.  In this case would you have confidence in the doctor’s findings? Of course not!

So, to apply this to psychometric tests let’s take an aptitude test. We’ve carried out research which confirms that a new numerical reasoning test can predict the performance of accountants. Those who score better on the test are rated as better accountants.  This is validity. The test is fit for the purpose of predicting accountant performance.  You will hopefully have full confidence in this finding if you know the test is reliable.  If however you expect the test is coming up with inconsistent scores for your candidates, it is unreliable, and, as in the scales example above, you will not have confidence in the test’s prediction of accountant performance. This is why reliability is a precursor to validity.

And why is all of this so important for this course?  It’s because you as the test administrator can enhance or reduce the reliability of the test by how you administer it in the first place.  Let’s now take a look at what factors you can and can’t influence in terms of reliability.

How psychometric test administrators can impact the reliability of tests

Factors Affecting Psychometric Test Reliability (C)2010 PsyAsia International: No Copying

Take a look at the graphic on the left. It shows different factors which can impact the reliability of psychometric tests. This applies to both aptitude tests and personality assessments.

Factors within the test

Generally, a test administrator is not responsible for this. The test publisher must design tests that will be highly reliable. Factors within the test means that the questions chosen must be accessible to all groups for whom the test is intended. If a subsection finds some questions difficult based on their group membership (i.e. non-native-English speaking groups may not understand a colloquialism used in a test question), then the test will be less reliable for that group. Although the publisher needs to ensure a reliable test, not all test publishers are reputable or know what they are doing! This is why the person who purchases the test needs to know how to evaluate it. We’ll show you later how to evaluate the test in greater detail.  Know for now that you do not evaluate a test or validate it by trialling it on yourself or your colleague as many untrained users think!

Factors within the respondent

Whilst the test administrator cannot control all the possible factors within a respondent, you can do your best to ensure you control for a much as possible.  It’s a good idea to think here about how you would like to be treated if you were undergoing a psychometric assessment for the first time. You’d probably like a friendly invitation letter explaining what is going to happen and why. You’d like to know that your data and results will remain confidential and only shared with decision-makers and only for the purpose that you’re undertaking the test. You’d also like to know what you need to bring with you and if possible, a few example questions as approved by the test publisher might help to set your mind at rest.  Finally it would be good to have a number to call should you have any special needs that you wish to convey to the administrators before the day.  So, when you arrive at the test centre you already know what is going to happen and why, you won’t be overly concerned, you’ll have all the right things with you (e.g., reading glasses) and you’ll know how long the session is going to last. If it’s a personality test you’ll be more likely to be open and honest because you know your results won’t go further than the selection or development committee and won’t be used for reasons beyond the reason you’ve already been given.

Ultimately here you are attempting to control for mood and expectations. Ideally you don’t want these to vary between candidates in order to give everybody the same start line.  On the actual day of the test you will go over all of these things again with the candidates in the room to ensure that they are all clear on what will happen and why.  Again, this sets the scene and mood, demonstrates your organisation’s “humanness” in the assessment process and provides candidates with an opportunity to ask questions.  Furthermore, on the day you will need to ensure that you administer the test instructions word for word and then administer the test exactly as intended by the test publisher. Doing all of this enhances consistency and thus increases reliability.  This is essential as we saw before because reliability is the precursor to validity.

Factors within the environment

How well would you be able to complete an aptitude test in a noisy room?  Or how about  room that’s freezing from too much air conditioning or too hot due to broken air conditioning?  Likewise, you need to ensure that the test environment is conducive to candidate performance each and every time.  This applies to personality assessment too. Although there is no right or wrong, your candidate will certainly feel more able to make an effort and respond accurately if you provide them with the right environment!  So, some time before the session you’ll need to check the room, make sure temperature controls work. On the day, switch them on in good time before the test so that by the time candidates arrive the room is just right.  Place a sign on the door to ensure you are not disturbed during the testing session and be sure to silence all phones in the room.  Candidates should of course have phones switched off too.  Ensure that once the session is over, all candidates leave at the same time so that they do not disturb others.  If a candidate really must make a restroom visit, they should be accompanied by an administrator and only one candidate at a time should go. Ensure that upon leaving and rejoining the room the candidate does not disturb others.
(Note: also a good idea to check there is no planned construction nearby and there are no fire drills scheduled on the day of testing. Do this before sending out your invitation to the candidate!)

Summary

By referring to these guidelines you’ll help to ensure that psychometric tests used by your organisation remain as reliable as the publisher intends them to be. By using short-cuts and not following the guidelines you’ll threaten the reliability and therefore the validity of the tests.  If you threaten a test’s validity it becomes unfit for purpose which means your company is wasting its money buying psychometric tools!

Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.

DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.

20 October 2010 – 27 October 2010 (weekend excluded) (5-day option also available)

 Conrad Centennial Hotel, Singapore

 **Please register early – previous courses have been fully booked up to 8 weeks before the start date**

 Introduction

Train for BPS Level A and B Certificates of Competence in Occupational Testing in Singapore with the company that won an international quality award in 2010; the company which brought regular BPS Certification training to Singapore; and the company which uses a British Psychological Society Award-Winning Dr. of Psychology to facilitate the course!

This intensive professional course provides training in psychometric assessment in the workplace, leading to the fulfilment of requirements for the British Psychological Society Certificates of Competence in Occupational Testing – BPS Levels A & B. After successful completion of the course, delegates are qualified to order, administer, score, interpret and feedback on a complete range of ability, aptitude and personality questionnaires. They will also be able to apply for the British Psychological Society’s Certificates which means the holder will be recognised internationally as having trained to the highest levels in the practical application of psychometric tests.

PsyAsia International is renowned by Human Resource professionals and organisational psychologists throughout Asia for being the leader in psychometric training in the region. Whilst many overuse the term “leader”, PsyAsia demonstrates their position with tangibles.  Recent innovations include moving of training to online mediums, blended learning for the BPS Level A and B and an online learning centre where delegates get immediate access to forums, chat with other delegates, virtual receptions with the facilitator and other course members, downloads of recorded training and more.

Dates:

20-22 October 2010: Level A

25-27 October 2010: Level B

Delegates may also opt to attend online training over 3 evenings instead of attending on 20 October, reducing total time away from the office to 5 days.  Furthermore, delegates may choose to undertake Level A in October and place Level B on hold until Q1 of 2011 for no additional fee.

Key Learning Objectives for Level A

Delegates will be able to:

• Register with all UK and many international test publishers to purchase tests and know what to look for and avoid in Singapore. With many poorly developed tests and non-psychologist distributors in Singapore this is crucial!
• Understand the range and types of psychometric tests available.
• Know when to use tests, and importantly – when not to.
• Decide which tests are appropriate for a particular situation, be it selection, development, coaching or team-building.
• Conduct a job analysis and identify the assessment requirements.
• Have an appreciation of other assessment methods such as assessment centres or behavioural interviews.
• Administer, score and interpret aptitude tests.
• Make informed decisions using psychometric test results.
• Provide quality feedback to clients and candidates.
• Integrate tests into a wider recruitment or assessment process.
• Have a sound grasp of statistical concepts used in testing.
• Appreciate the importance of test reliability and validity and be able to assess it.

Key Learning Objectives for Level B

Delegates will be able to:

• Increase their awareness and understanding of individual differences between people.
• Integrate the use of personality assessments into selection and development activities.
• Decide which questionnaires are appropriate for particular situations.
• Administer, score and interpret personality data.
• Provide professional selection feedback to both candidates and recruiting customers.
• Coach individuals to develop using personality information.
• Use personality questionnaire information to support structured interviewing.
• Critically assess the value of different tools for specific purposes.
• Identify and reject inflated claims of validity – so-called ‘Barnum’ or ‘horoscope’ effects.
• Relate personality factors to organisations’ core competencies.
• Design and write selection reports for recruiting clients / managers.
• Design and write development reports to aid self-awareness of clients / managers.
• Maintain appropriate records and data for audit, research and feedback purposes.
• Demonstrate professional Best Practice and compliance with employment legislation.

See the course webpage for a full day-by-day syllabus.

No Exams!

At the current time it is up to individual course providers to decide how to assess their delegates; some may use exams whilst others use workbooks and other assessments. At the current time, there is no exam requirement for our Level A and B training in Singapore; we find that our delegates prefer this. Instead we assess via a workbook and practical exercises.

Facilitator
This course is facilitated by PsyAsia International’s resident director, Dr. Graham Tyler. Dr. Tyler is well known in Asia for his expertise and work in psychometrics. His PhD was by way of a double-scholarship award from the Australian Government and focussed on psychometric assessment, personality and performance at work in Asia. Dr. Tyler is a fully Registered Organisational Psychologist, a Chartered Scientist, a consultant/trainer to numerous Singapore Government departments and Fortune 500 companies and a university lecturer. He is a published author in Singapore-based publications (Straits Times, Human Capital Plus, Human Resources) and peer-reviewed journals such as the International Journal of Selection and Assessment, the BPS Selection and Development Review and The Occupational Psychologist. He is a reviewer for scientific journals and conferences and a supervisor for Intern Psychologists on behalf of Australian and local Universities. Dr. Tyler was the first to bring regular training leading to the British Psychological Society’s Certificates of Competence in Occupational Testing to Singapore in 2002 having introduced them to Dubai prior to that. His work with Psychometric Tests has received recognition from the British Psychological Society with an award for Scientific Contribution to Psychology. Plentiful delegate reviews of Dr. Tyler’s training can be seen at the course webpage.

Additional Benefits

The course facilitator and our psychologist consultants are based locally, this means that we can blend the BPS syllabus with knowledge and experience of local laws, culture and practices and that even when the course ends expert psychologists will be nearby to offer ongoing support. Given that we are not a training reseller, we don’t need to ship in overseas facilitators. As a result, we can be flexible in terms of our course offerings. Delegates can choose which parts of the course to sit and when – no need to attend all 6 days at once! You may attend Level A in October and then Level B in 2011. You can even start out by attending our 2-day Identity Accreditation Course or study our Online Test Administration Course and have those count towards your Level A/B! Finally, because we are local there is no need to send your follow-up work overseas for marking. You can essentially finish Level B as soon as you are ready – and don’t hang around for your results – we guarantee to mark your work within 10 days, although most is marked within 2-3 days of receipt – you can check our online performance review for recent dates!

Furthermore, because we are not a test publisher, our training is unbiased and we introduce many tests (both good and bad) to delegates. In addition to this, for those delegates who wish to study and be accredited in numerous tests, we offer 50% discount off our Saville Consulting Wave® one-day training which follows Level B (60% if you book before 30 July). Of course, we include Identity Questionnaire and Apollo Profile accreditation in the A/B fee as standard for all delegates. That’s not all! In addition to the SG$1200+ of materials we give you on the course (which include an online test system with starter credits), all delegates will receive a 30% discount on any Identity Personality Assessment products purchased within 1 month of their qualification and a 10% discount on all other non-discounted products which we offer.  There’s even more, but we’ll save some surprises for the course!

More details

The Psychometric Assessment at Work Course in Singapore
http://www.psyasia.com/psychometric_test_training_course.php

Other Psychometric Training Courses in Singapore from PsyAsia International
http://www.psyasia.com/psychometric_training.php

Please follow the links at the web pages to register online. For any queries not addressed at the website, kindly call us on 6329 9629 or email us via http://www.psyasia.com/email

Visit the website

Saville Consulting Wave® – Highest Validity per 15 minutes of test-time!

The Saville Consulting Wave was developed by the originator of the OPQ and co-founder of SHL, Professor Peter Saville as an alternative to the static assessments available in the market. Wave assesses candidate’s motives and talents within a validation-centric framework. It was validated within 100 separate businesses.

The tool uses both normative and dynamic ipsative questions and has been shown to have greater validity in predicting performance and leadership than any tool it has been compared to in research. Furthermore, the shorter version of Wave, the Focus Questionnaire has more validity per 15 minutes of test-taking time than any comparison questionnaire.

Profile jobs, run 360 performance appraisal, assess personality type for team-building, assess entrepreneurial potential and provide extensive development advice all within one framework

The performance culture framework which underlies the Saville Consulting Wave allows the tool to be used extensively for different HR applications from recruitment/selection through to performance appraisal and development.
Follow the links below for further details:

Wave Professional Styles	Performance Card Set
Wave Focus Styles	Wave Performance 360
Wave Types	Wave Development
Wave Entrepreneurial	Wave Culture
Wave Job Profiler

View some sample reports from Wave here

For the early-bird offer (25% discount), please register at http://www.psyasia.com/register
and quote WAVESEB for Singapore courses and WAVEHEB
for Hong Kong Courses. Deadline is 15 May for Singapore and 31 May for Hong
Kong.

Note – delegates on our BPS Level B course in
Singapore always get 50% discount off the Wave Conversion
course which follows their Level B training. More details at http://www.psyasia.com/bpscourses

Course Reviews from Previous Attendees

“Informative, relevant to work, knowledgeable facilitator”

Danny
Hay Group

“Good introduction to the tool and practical session was useful”

Geraldyn Fong
Managing Consultant (Psychologist)

Hudson Global Resources, Singapore

“Insightful and informative. The methodology behind the Saville
Consulting Wave Report is light years ahead of other psychometric tests yet
it is a breeze to use! The interface between motives, competencies and culture
is exactly the missing link that recruiters are looking for.”

Diana Yeo
Ministry of Defence, Singapore

“A highly practical and enjoyable approach to the application
of an extremely useful tool for selection and development – well worth the investment,
highly recommended.”

Neville Pearsall
Director HR
RMIT International University, Vietnam

Not ready for training and accreditation
but still want to use the Wave?

Then please consider PsyAsia’s Psychologist-on-Call™
service instead. Our registered psychologists will take care of the complete process for you, including a call and behavioural interview for your candidates and a feedback session with the decision-maker. More details at http://www.psyasia.com/psychologist_candidate_screening.php.

PsyAsia International is pleased to announce that until the end of March we will be offering free daily webinars to showcase our product range. Their will be no set agenda. The agenda will be set by attendees. Please note however that product knowledge may differ depending on which of our consultants is running the webinar. Come along and chat with our consultants, see the Saville Consulting Wave, Identity Personality Assessment and the Apollo Profile in action. Ask questions about training and consulting options and so forth!

For times and to register, please click here…

Types of Bias in Psychometric Test Translation

With the demand and need for psychological tests increasing in various different cultures and countries, there has been much greater awareness regarding some of the issues that are associated with the development or adaptation of tests to be used in contexts and situations that may be different from which the test was developed for. This article focuses on one of the key aspects of translating tests, the types of bias that can occur.

When utilizing the test in a new cultural group, it is not quite as simple as directly translating the test, administering it and then comparing the results for its validity. There are a number of issues that need to be considered such as whether the area assessed with the test applies to the new culture or whether is may be biased towards that group and whether what is assessed by the test also has similar behavioral indicators? These are just some of the potential areas where bias can be found in the translation of tests and affect the validity of the test being utilized in the new context.

Van der Vijer & Hambleton (1996) differentiates between three distinct types of bias that may affect the validity of tests that have been adapted for different cultural contexts and these are construct bias, method bias and item bias.

Construct bias occurs when the construct (e.g. personality) that is measured by the test displays significant differences between the original culture for which it was developed and the new culture where it is going to be utilized. These differences can occur in the way that the construct was formulated and developed as well as in the relevant behaviors that are associated with the construct. It is critical to examine whether the underlying theory of the test is subject to construct bias and this can be examined through the studies examining the construct and its associated behaviors in the context that it will be utilized in. If there are significant differences found in these studies, it may indicative that there is construct bias. Major revisions may be required to overcome this bias. If not, the validity of the test will be affected.

Method bias refers to factors or issues related to the administration of the test that may affect the validity of the test. Examples of areas that method bias can occur include social desirability, acquiescence response styles, the conditions in which the test was conducted and the motivation of the respondents. Across cultures, there potentially can be differences that can occur in these areas and these can affect the way that the respondents answer the items in the test. This potentially may lead to differences between found that can be erroneously attributed to cultural differences when in fact, these differences are the result of differences in the administration procedures. As a result, it is threat to the validity of tests that have been adapted for use in new cultures. Test developers also not only need to focus on the adaptation of the test itself but also need to be aware of issues regarding the implementation of the test in a new context.

Item bias is another source of bias that can occur in the translation of tests and these refer to biases that occur with the items in the test. This is usually the result of either poor translation choices for items or due to culturally inappropriate translations. For example, the phrase “kick the bucket” is essentially a phrase that referring to passing away in the Western context and is commonly known by most people in that culture; unfortunately, this phrase would have no meaning for people from cultures without any prior experience with that phrase. In this manner, a literal translation of that phrase would be a poor translation as it does not convey the correct meaning of the item. The items in the test need to be culturally equivalent, where the meaning of the items needs to be correctly translated so as to maintain the validity of the test in the new cultural context.

These are some of the biases that may occur during the translation of tests. Test developers will need to be aware of the sources of bias and take the appropriate measures to avoid these biases.

References:

Van der Vijer, F. and Hambleton, R. K. (1996). Translating tests: some practical guidelines. European Psychologist, 1, 89-99.

Psychometric Training in Singapore, Hong Kong, Malaysia, and China
If you are serious about using psychometric tests properly then we recommend joining PsyAsia International’s Psychometric Assessment at Work Course which leads to a certificate of competence in Occupational Testing Level A and Level B from the British Psychological Society. The Course is run publically in Singapore and Hong Kong or in-house anywhere.
More details about BPS Level A and B in Singapore and Hong Kong

Online Psychometric Training – Worldwide
Alternatively, you might be interested in introductory Online Psychometric Test Training presented live by a registered psychologist. PsyAsia is offering a special fee of just US$12 for anybody who registers for the February online psychometric training course!
More details about online psychometric test training

Identity Psychometric Personality Questionnaire Webinar

-Overview of the Identity Self Perception Questionnaire
-Scales of Identity
-Derived Psychological Models
-Identity Reports
-Identity Online
-Benefits of Identity
-Applications of Identity
-Comparison with other leading questionnaires
-Identity Accreditation

Click here to register

To complement our BPS Level A and BPS Level B Certificates of Competence in Occupational Testing Training in Singapore, PsyAsia International has announced new dates for the same psychometric training course in Hong Kong. PsyAsia has been running this course in Singapore and Hong Kong since 2002 and has collected many impressive delegate reviews. The course is run by a British Psychological Society award-winning resident and registered business psychologist with a PhD earned through work with validation of Psychometric Tests in Asia. For more details of the course and facilitator and to register online, please click here.

The Hong Kong Psychometric Assessment at Work course will run over the following dates: Level A: 9-11 March 2010 Level B: 16-18 March 2010 Psychometric Test Administration only: 9 March 2010

The Singapore Psychometric Assessment at Work course will run over the following dates: Level A: 27-29 January 2010 or 24-26 February 2010 Level B: 1-3 March 2010 Psychometric Test Administration only: 27 January 2010 or 24 February 2010

Delegates may opt to join Level A in Singapore and Level B in Hong Kong if they wish at no additional cost but should contact us before booking in order to ensure availability.

The day immediately following Level B is conversion training for the Saville Consulting Wave^®. Level B delegates may register for this training at 50% off the regular fee by selecting that option during the booking process.

PsyAsia International, Asia’s leading independent provider of psychometric assessment products and training today announced a free psychometric test training workshop in three prime Asian locations in December.  The workshops will cover the basics in psychometric assessment, such as reliability, validity and error, along with pointers and discussion on how to choose the right psychometric test.  Courses will be facilitated by PsyAsia’s organisational psychologists who have at least an MSc in Organisational Psychology along with the British Psychological Society Level A and B Certificates of Competence in Occupational Testing.  PsyAsia’s psychometric training courses in Asia have been the subject of many years of positive reviews (many published at the website).  This course will bring best practice and an introduction to competence to those who would otherwise not realise the issues involved in testing due to the costs of regular training courses.

PsyAsia will charge registrants a small fee to reflect the company’s costs in running the training session.  The fee is just US$70 (SG$99/HK$550).  This covers a full buffet lunch in a top hotel along with day-long refreshments, handouts and a certificate of attendance.  However, PsyAsia will even refund this fee back to any participant who registers for one of the company’s psychometric accreditation courses within 6 months!

The dates are:
Hong Kong: 3 December 2009
Kuala Lumpur: 8 December 2009
Singapore: 9 December 2009

More information
Online Registration