Available online on 15.07.2021 at http://jddtonline.info

Journal of Drug Delivery and Therapeutics

Open Access to Pharmaceutical and Medical Research

Copyright  © 2021 The  Author(s): This is an open-access article distributed under the terms of the CC BY-NC 4.0 which permits unrestricted use, distribution, and reproduction in any medium for non-commercial use provided the original author and source are credited

Open Access  Full Text Article                                                                                                                                         Research Article 

Content Validity of Drug Addiction Recovery Test Instruments Using Content Validity Ratio (CVR) Method

Norashida, S. R., Norshahira, O., Lukman, Z. M.

Faculty of Applied Social Science, Universiti Sultan Zainal Abidin, Kuala Nerus, 20300, Terengganu, Malaysia

INTRODUCTION:

Drug recovery is personal and a process of individual change that focuses more on attitudes, values, goals, skills and roles .^1,2 Recovery also refers to a new meaning in one's life after successfully overcoming the 'tragedy' of drug addiction. ^3,4 Being able to cope with addiction can mean that drug addicts are going through a difficult phase and this can lead to better change in their lives. Drug addicts often want to stay in recovery by finding suitable and loved jobs in the community to continue their lives. ⁵ There had three stages of recovery are early recovery, middle recovery and late recovery.⁶ Although the role of institutional orientation in drug rehabilitation is known to be important, there are internal and external factors that have been identified as influencing addiction recovery processes. Addiction recovery is different according to the modules and programs that have been set for each rehabilitation centre in Malaysia. In this country, rehabilitation centre are divided into two types of centre: government rehabilitation centre and private drug rehabilitation centre. These institutions are helping drug addicts cope with the programs and modules that have been provided. The process of addiction recovery is based on the type of drug used based on the behaviour of drug addicts on the behaviour of drug addicts as a result of drug addiction.⁷ A Drug Addiction Recovery Assessment Instrument is to measure the stage of drug addiction recovery among drug addicts based on the context of Malaysia environment. This study helps the government under the National Anti-Drugs Agency (NADA) and Private Drug Rehabilitation Centre to develop instrument and analyse the stage of addiction recovery in drug addict. This study highlights four component namely drug dependence, drug possible relapse, drug resiliency and client's mental strength based on previous research.

Validity generally means the ability to measure what is to be measured and is one of the important aspects in an instrument. ⁸ Without satisfactory validity, the psychometric characteristics of an instrument will be affected even though the instrument has excellent reliability. ^{9, 10} The validation process will guarantee the instrument to have properties that are defensible, accurate, appropriate, meaningful and useful. ^{10, 11}    Therefore, the validation process must be done precisely in order to develop a valid instrument to use. In general, validity is divided into several categories that have different purposes and goals namely face validity, content validity, criterion validity and construct validity. ¹² The CVR method has also been widely used by local and foreign researchers as a preliminary step in the instrument preparation process. ^{13, 14}

According to ¹⁵, identified content validity as an adequate level of instrumentation to build the focus of the study. In this section, the developed questionnaires were validated by eight experts selected according to their experience in a particular field of research. The experts consist of Universiti Sultan Zainal Abidin (UniSZA), Universiti Malaysia Terengganu (UMT) and Universiti Pendidikan Sultan Idris (UPSI). The instrument was then examined, reviewed and assessed for its accuracy as well as examining the fundamental factors of each item constructed by the researcher. To identify whether an item measures construction or not, it requires evidence. Evidence can be shown through the content validation procedure. Content validation can be described as an assessment of how accurate a test sample is .^{15, 16} According to ¹⁷ proposed several procedures for obtaining content validity, relevant to each construction and also involving the construction of study instrument items.

MATERIALS AND METHOD:

This study uses a quantitative approach using questionnaires as research instruments. Respondents for this study were comprised from experts in related fields has been identified as capable of bean evaluator for the item review process in detail. This study divides experts into two categories namely professional specialists and field experts. ^{18, 19} A professional is an expert directly involved in the study or ever publish temporary related articles field experts in turn are experts who have skills or experience specifically in fields studied .¹⁸ Because this study deals with addiction recovery, then experts are made up of individuals who are involved directly in the innovation of their respective fields. This study has set several criteria which must be met for the selection of specialists. For data collection purposes, this study using a variety of approaches that is face to face (direct approach), by post (postal survey) and online (email/internet survey) according to the comfort and needs of the specialist. ²⁰ In the early stages, specialists are contacted via email to obtain consent at in addition to explaining the purpose and procedure of the study. The study involved only eight experts to evaluate each of the items according to the DART component. 

RESULT AND DISCUSSION: 

Result 

In this study, content validity was determined using Content Validity Ratio (CVR) by assessing the level of importance of each item based on three scales namely Essential (very important), Useful but not essential (useful but not important) and Not necessary (no need). To determine the validity of the content, calculations should be performed using the formula CVR = [ne- (N/2)]/(N/2). This formula explains that CVR refers to the value of constructed items, ne is the number of expert panels that rated the item as essential and N is the total number of expert panels involved (N = 8).

According to ²¹, CVR values are in the range of -1 to +1. A +1 value indicates that the item rated by the expert panel is important in the validity of the content. If the CVR value is ˂0, it indicates that less than half of the expert panel rated the item as essential. If the CVR value = 0 indicates that part of a group of experts is involved in evaluating the item as not important and another part evaluates it as essential. While the CVR˃0 value indicates that more than half of the expert panel rated the item as essential. The higher the value from 0, the higher the validity of the content.

Thus, if CVR = 1, clearly indicates that all expert panels have agreed to evaluate the item as essential (essential) and then have a high content validity. The findings of the study were determined through psychometric tests through CVR values that were set based on a total of 8 expert panels of 0.75. ²¹ After testing was done, a total of 68 items were identified to be retained while 12 other items with values below 0.75 were re -evaluated and refined. According to, ²¹ suggest two approaches that can be taken i.e. whether the item is purified or dropped directly from the instrument. Table 1, Table 2, Table 3, and Table 4 below, show a summary that measures the evaluation gain from 8 expert panels along with 80 items of DART instruments according to components that have been retained and re -evaluated based on the CVR technique by. ²¹

Through the determination of CVR values based on expert assessment, Table 5 below, show the overall distribution of items retained was 68 items and 12 items improved which included drug dependence component (18 items) with 2 item repairs, possible drug relapse (17 items) with 3 item repairs and drug resilience (16 items) with 4 item repairs. While the client's mental strength (17 items) with 3 item repair.

Overall, the results of the study show the quality and robustness in the instrument validation process that has been done. The variation of study findings and detailed feedback from experts for the entire 80 initial items clearly indicates that the expert panel has conducted a thorough and thorough content validation process to help produce items that are potential and suitable for use in a given context. The carefully conducted content validation process has also succeeded in providing a more concrete content validity value. Expert feedback indicated 12 items with low CVR values tended to be problematic. The panel of experts raised two key issues that deserve attention. The first feedback raised by the experts was related to the drug dependence component i.e. items 3 and 15 which were unclear and difficult to understand. Items 3 and 15 are acknowledged by experts to be quite important but some improvements need to be made if they want to be maintained so that the instrument remains of good quality. As for items 2, 6, and 9 for the relapse possibility component experts think the item is rather elusive. 

The sentences used are too long in items 2,6,9 quite confusing and experts suggest that the sentences be shortened so that they are easy to understand. For items 2 and 9 of the drug resilience component the expert suggested that the sentence structure be re -evaluated as it was somewhat confusing even though the items were stated in brief sentences. While for items 10 and 18 it is not clear and difficult to understand and some improvements need to be made if it is to be maintained. For item 9 in the mental strength component the expert suggested that one of the words “unloved” be removed from the existing sentence. As for item 14 in the mental strength component, the majority of experts think that the word “work” found in the sentence is removed from the existing sentence and replaced with a more appropriate word. For item 15 it is not clear and difficult to understand and some improvements need to be made if it is to be maintained.

Items were reviewed after final agreement with the panel and supervisors. This includes language improvements and corrections in line with what has been discussed in the framework. Initially, 77 items were selected in the instrument; however the word redundancy resulted in the omission of some items that had been described in the previous section, and was left with only 70 items in the instrument. The item review process takes about a month before it is completed. Thereafter, it is submitted to the supervisory committee so that they can analyse for the proposed modifications to ensure they are in line with the objectives in this area of study. Therefore, in conclusion a total of 80 items of the DART instrument were successfully retained prior to conducting the study.

Table 1: CVR Values of Drug Dependency Component Items

* The number of expert panels that have rated the item as essential. ** Content Validity Ratio (CVR) = (Ne-N/2)/(N/2) involved eight expert panels (N = 8), items with CVR values of 0.75 and above were retained as instruments. While the CVR value that is less than that value has been re -evaluated.

Table 2: CVR Value of Drug Possible Relapse Component Item

* The number of expert panels that have rated the item as essential. ** Content Validity Ratio (CVR) = (Ne-  N/2)/(N/2) involved eight expert panels (N = 8), items with CVR values of 0.75 and above were retained as instruments. While the CVR value that is less than that value has been re -evaluated.

Table 3: CVR Value of Drug Resiliency Component Items

* The number of expert panels that have rated the item as essential. ** Content Validity Ratio (CVR) = (Ne - N/2)/(N/2) involved eight expert panels (N = 8), items with CVR values of 0.75 and above were retained as instruments. While the CVR value that is less than that value has been re evaluated

Table 4: CVR Value of Client Mental Strength Component Item

* Number of expert panels that have rated the item as essential. ** Content Validity Ratio (CVR) = (Ne-N/2)/(N/2) involved eight expert panels (N = 8), items with CVR values of 0.75 and above were retained as instruments. While the CVR value that is less than that value has been dropped.

Table 5: Distribution of Items after Expert Verification and CVR Test

DISCUSSION

The CVR method used in this study also provides quality assurance in the content validation process performed due to the use of various expert variations in terms of number and background. The diversity of experts involved in this study has indirectly ensured the validation process performed as well as reassured others that the instrument is valid, clear and reflects the measurements to be performed. ^{22 23} The validity of the content conducted for the findings from experts using the CVR method in this study has provided important information about the validity of the content of the drug addiction recovery test instrument. The emphasis on the importance of each item in the CVR method has helped to empirically filter each item through quantitative procedures to ensure that each item truly represents the content of the component domain. ²⁴ High CVR scores for the majority of items indicate the stability and robustness of the instrument to measure what is to be measured from an expert’s point of view. Strong evidence of this expert consensus has helped increase confidence in the content of the instrument. ²² The CVR method has also helped to analyse the quantitative data of expert agreement more efficiently to provide strong evidence for the decision to retain or drop items in the instrument. Problematic items can be easily identified and supported by solid evidence.

However, the process of refining and improving these items will be carried out not only on the 12 problematic items, but on all items based on expert comments and recommendations. All feedback received will be refined and given due attention by the researcher. The updated items will be prepared for a pilot study. Based on the findings obtained, this study can also confirm that the drug addiction recovery test instrument is based on four components, namely drug dependency, drug possible relapse, drug resiliency and client mental strength. All experts agree that the majority of items from this component are important and appropriate for measuring the degree of addictive recovery in rehabilitation centres. No addition of components or items proposed indicates that the instrument is consistent and suitable for use.

Summarizing results from experts stating the majority of items are appropriate also serve as evidence that the instrument is relevant to the context of the study. Remarks and feedback on the instrument focused only on sentence structure and item repetition. No major errors such as irrelevant items or inappropriate components were identified in this study indicating the instrument was on track. The consensus of experts who suggested that the next action be taken, namely a pilot study, is also a clear indicator that this instrument is suitable and relevant for use in the study of addiction recovery in Malaysia.

CONCLUSION:

The main goal of this study was to determine the validity of the content and see the suitability of the drug addiction recovery test instrument in drug addicts in rehabilitation centres. Once the validation process was performed, only 12 items were reported to have CVR values lower than the critical values. The findings of this study clearly prove that the instrument is suitable for studying the level of drug addiction recovery because of its validity and having an appropriate function, especially in the context of addiction recovery in Malaysia. Based on these preliminary findings, the researchers believe that this instrument has the potential to be a valid and reliable instrument for measuring the level of recovery of drug addiction among drug addicts in rehabilitation centres,

The CVR method used has provided empirically strong evidence to confirm the validity of the instrument based on expert consensus. The decision to maintain, improve or refine items in the instrument can be made confidently and clearly after analysis using CVR is performed. In conclusion, this study has proved that the drug addiction recovery test instrument that has been constructed has high content validity and is suitable for use on drug addicts who are in rehabilitation centres in the future. However, it is important to note that this instrument is still in development and needs further study specially to assess its psychometric characteristics before being used in actual studies. Therefore, the next step in the development of this instrument is to conduct a pilot test to study the reliability of this instrument. It is recommended that all 80 items be refined to undergo a pilot study and analysed using statistical analysis such as factor analysis so that the items can be analysed in more depth.

ACKNOWLEDGEMENT:

We would like to thank the participants in this study.

REFERENCES

1. De-Micheli, D. and Formigoni, M. L., Screening of Drug Use in a Teenage Brazilian Sample Using the Drug Use Screening Inventory., Journal of Addictive Behavior, 2000; 25(5):683691. https://doi.org/10.1016/S0306-4603(00)00065-4

2. Anthony, W.A., Recovery from Mental Illness: The Guiding Vision of the Mental Health System in the 1990s, Innovations and Research., 1993; 2:17-24. https://doi.org/10.1037/h0095655

3. Deegan, P. E., Recovery: The Lived Experience of Rehabilitation, Journal of Psychiatric Rehabilitation, 1988; 11:11-19. https://doi.org/10.1037/h0099565

4. Miller, A. F., Substance Abuse Treatment for Women with Children. Corrections Today, 2001; 6:88-92.

5. Mental Health Advocacy Coalition, Destination: Recovery. Auckland, 2008, Mental Health Foundation of New Zealand.

6. Matokrem, L., Intervensi dan Peranan Kaunselor untuk Menjana Kepulihan Klien Sepanjang Hayat, Jurnal Antidadah Malaysia, 2007, 1 Jun 2007.

7. WHO, Scientific Group on the Evaluation of Dependence Producing Drugs Wld Hlth Org. techn, 1964, Rep. Ser., 287.

8. Field, A. Discovering Statistics Using IBM SPSS Statistics. 5th Ed., London: Sage Publications Ltd, 2018.

9. DeVellis, R. F., Scale Development Theory and Applications. 4th Ed. London: SAGE Publication Inc, 2017.

10. Furr, R. M., Scale Construction and Psychometrics for Social and Personality Psychology. London: SAGE Publications Inc. 2014.

11. Ghazali Darusalam & Sufean Hussin. Metodologi Penyelidikan Dalam Pendidikan: Amalan Dan Analisis Kajian. Kuala Lumpur: Penerbit Universiti Malaya. 2016.

12. Taherdoost, H., Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research, International Journal of Academic Research in Management (IJARM), 2016; 5(3):28-36. https://doi.org/10.2139/ssrn.3205040

13. Doustmohammadian, A., Omidvar, N., KeshavarzMohammadi, N., Abdollahi, M., Amini, M. and Eini Zinab, H., Developing and validating a scale to measure Food and Nutrition Literacy (FNLIT) in elementary school children in Iran., PLoS ONE 2017; 12(6):1-18. https://doi.org/10.1371/journal.pone.0179196

14. Mohd Effendi Ewan Mohd Matore, Hisyamsani Idris, Normawati Abdul Rahman and Ahmad Zamri Khairani. Kesahan Kandungan Pakar Instrumen IKBAR Bagi Pengukuran AQ Menggunakan Nisbah Kesahan Kandungan. Proseeding of International Conference On Global Education V (ICGE V) (May), 2017; 979-997.

15. Delgado, R. E., Carretero-Dios, H., and Ruch, W., Content Validity Evidences in Test Development: An Applied Perspective. International Journal of Clinical and Health Psychology, 2012; 12(3):449-460.

16. Cohen, R. J. and Swerdlik, M. E., Psychological Testing and Assessment: An Introduction to Tests and Measurement. 9th ed. New York: McGraw Hill. 2013.

17. Rico, E. D., Dios, H. C., and Ruch, W. Content validity evidences in test development: an applied perspective. International Journal of Clinical and Health Psychology, 2012; 12(3):449-460.

18. Rubio, D. M. G., Berg-Weger, M., Tebb, S. S., Lee, E. S. and Rauch, S. Objectifyng content validity: Conducting a content validity study in social work research, Social Work Research, 2003; 27(2):94-104. https://doi.org/10.1093/swr/27.2.94

19. Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H. and Nikanfar, A. R, Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication, Journal of Caring Sciences, 2015; 4(2):165-178. https://doi.org/10.15171/jcs.2015.017

20. Brinkman, W. P., Design of a questionnaire instrument handbook of mobile technology research methods. Handbook of Mobile Technology Research Methods, 2009; 31-57.

21. Lawshe, C. H. A quantitative approach to content validity. Personnel Psychology, 1975; 28(4):563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

22. George, N., Barrett, N., McPeake, L., Goett, R., Anderson, K. & Baird, J. Content validation of a novel screening tool to identify emergency department patients with significant palliative care needs. Academic Emergency Medicine, 2015; 22(7):823-837. https://doi.org/10.1111/acem.12710

23. Yaghmaie, F., Content validity and its estimation. Journal of Medical Education, 2009; 3(1):870.

24. Almanasreh, E., Moles, R. & Chen, T. F. Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 2019; 15(2):214-221. https://doi.org/10.1016/j.sapharm.2018.03.066