Designing data collection tools for psychiatric research: a practical guide for postgraduate psychiatry trainees

INTRODUCTION

Most psychiatric research rests on information collected from participants. Common variables include physical parameters, cognitive task results, and a wide range of psychometric instruments measuring constructs such as emotion, personality, and attachment. Different study designs and research questions require tools capable of capturing appropriate data for that specific design and question.

Data collection is an irreversible step-errors in tool design cannot be corrected once recruitment has begun without compromising the dataset. Adequate time must therefore be devoted to designing a tool suited to the specific study design before data collection begins. The data collection tool should be finalised before ethics submission, as any substantive modification after approval requires a formal amendment application. Common consequences of poor design are summarised in Table 1.

MATCHING THE TOOL TO THE STUDY DESIGN

The structure of a data collection tool should reflect the demands of the study design. Table 2 outlines key considerations by study type. In general, cross sectional study design is the most commonly used in psychiatric and-psychology-research.

*This guide does not cover instrument development, translation methodology, or psychometric validation, or designing online data collection tools, each of which constitutes a study in its own right. This is an opinion and practical suggestion, based on personal experiences, which addresses the practical decisions involved in assembling a tool from existing validated instruments and locally designed components.

*RCT-Randomised controlled trail

ACCURACY WHEN COPYING SCALE ITEMS

Research tools are frequently assembled by copying (or manually typing) scale items from published papers, scanned originals, translated documents, or prior studies. This process may introduce errors: items may be inadvertently truncated, words may be dropped or altered, response anchors may shift, and item numbers may be displaced - all without the researcher noticing, because the resulting text might still read as plausible. An item that has lost a negation, gained a word, or had its anchor labels swapped is no longer the validated item; it is a modified item with unknown psychometric properties. To guard against this, each scale in the assembled tool should be compared word-for-word against the validated source document before finalisation, not merely read through for general sense. Where the original instrument exists in a published or official format, that version should be the reference, not a version copied from another study's appendix, which may itself contain errors.

TOOL LENGTH AND PARTICIPANT BURDEN

There is no universally accepted upper limit on the number of scales or items a research tool may contain, but the guiding principle is that every scale must be independently justified by the research question. Including scales because they are commonly used, freely available, or tangentially interesting - sometimes called data dredging - inflates participant burden without proportionate scientific return and increases the risk of incomplete responses, random responding, and early dropout. In a narrative review addressing the optimum length of a questionnaire, 25–30 questions and a completion time of 30 minutes were suggested.⁴ A self-administered tool taking longer than 45 to 60 minutes is likely to cause significant fatigue in most outpatient and community samples; inpatient or acutely unwell populations may tolerate considerably less. Estimated completion time should be calculated during pilot testing rather than assumed.⁵

MODE OF ADMINISTRATION (INTERVIEWER-ADMINISTERED VERSUS SELF-ADMINISTERED FORMATS)

The choice of administration format is a design decision that should be made deliberately, documented in the protocol, and reflected consistently in the tool layout (Table 3).

*HDRS – Hamilton Depression Rating Scale, PANSS – Positive and Negative Syndrome Scale, CGI – Clinical Global Impression

*ACE-IQ – Adverse Childhood Experiences International Questionnaire, MCQ-30 – Metacognitions Questionnaire-30

FORMATION OF ADMINISTRATIONS (ONLINE VS. PAPER-BASED)

This guide focuses on paper-based tools. Online data collection requires separate consideration, including issues of format validation and psychometric equivalence. Reader is directed to other source for designing online data collection tool.⁶ For paper-based data collection tool, participant may require almost no resources, and does not require digital literacy. For in-patient or out-patient, with limited digital literacy, and when participant might need support in competing the data collection tool, paper-based tool might be preferred. Manual data entry from paper-based form may introduce errors. For large community-based survey, when digital literacy may not be an issue, online data collection tools are widely used. Limited skip logic may be available based on online data collection platform, which can be used by indicating arrows or instruction in paper-based forms.

LANGUAGE

In general, one participant should receive the tool in one language only. Mixing languages within a single participant's tool should not be done unless explicitly justified in the methodology - for example, in a language validation study. Two separate language versions of the tool should be prepared rather than placing two languages side by side within a single document. It is important to note that different language versions are considered data-equivalent only when formal cross-cultural validation has been demonstrated for each version.⁷

PHYSICAL FORMAT

The physical appearance of the tool affects participant compliance and data quality. Generally, A4 portrait paper may be used commonly, but for complex grid, visual analogue scale, branched logic, A3, or booklet format too can be adapted. Clear and easy to read font of appropriate size should be used; decorative fonts should be avoided and professional appearance should be maintained). To improve readability, adequate white space should be kept in between the questions and scales. For qualitative research adequate space for documentation of responses should be provided.

STRUCTURAL ORGANISATION OF THE TOOL

The following sequence is generally recommended for paper-based psychiatric research tools. (Table 4) Modifications may be necessary depending on the study design, but the information sheet and informed consent form should almost always appear first, given their ethical and legal requirements. The general ordering principle - easy before difficult, neutral before emotionally activating - should be followed. Within the sociodemographic section, sensitive items such as religion, caste, and financial details can be presented last.

SOCIODEMOGRAPHIC DATA

Sociodemographic data should be limited to variables that are analytically relevant to the study. Blanket data collection - gathering variables simply because they are routinely collected - should be avoided. Where possible, data should be collected in continuous or ordinal format rather than categorical, as continuous data permit a wider range of analytic approaches. For example, age in years, years of formal education, total duration of illness, and duration of untreated illness in months are all preferable to their categorical equivalents.^8,9 For items that may be ambiguous, brief explanatory notes should be provided within the form - for instance, clarifying the definition of occupation as used in the Kuppuswamy scale, or explaining how residential status should be interpreted.

*ACE-IQ- Adverse Childhood Experiences International Questionnaire (WHO).

*VAS – Visual Analogue Scale, PHQ-9 – Patient Health Questionnaire-9, HDRS – Hamilton Depression Rating Scale, MADRS – Montgomery-Åsberg Depression Rating Scale

SCALES AND STRUCTURED INSTRUMENTS

Scale sequencing within the tool should follow the same principles applied to the overall tool structure. (Table 5).

CRITICAL RULES FOR VALIDATED SCALES

Validated scales must be used as developed - altering item order, response format, language, or mode of administration without revalidation compromises psychometric properties. Trainees should also use scales validated for the specific language, population, and clinical condition relevant to their study, rather than defaulting to scales validated in a different population simply because they share the same language (Table 6).⁷

INSTRUCTIONS

Instructions should be provided at two levels - global and section-specific - as outlined in Table 7. Instructions must be visually distinct from item text so that participants do not confuse them with response items.

CODING AND DATA ENTRY COMPATIBILITY

The data entry spreadsheet and coding scheme should be designed simultaneously with the data collection tool, before recruitment begins. Attempting to construct a coding system retrospectively reliably produces confusion, inconsistency, and avoidable errors.⁹ A consistent missing data convention must be established before entry begins and applied across the entire dataset.¹⁰ (Table 8)

SPECIAL CONSIDERATIONS IN PSYCHIATRIC POPULATIONS

Data collection in psychiatric populations requires particular care. Where sensitive or distressing content is included, contact details for available helplines or support services should be provided within the tool as part of standard ethical requirements (Table 9) ^11-13

PILOT TESTING

Pilot testing the assembled tool is recommended before recruitment begins. The pilot should involve both knowledgeable colleagues and, wherever possible, participants from the target population - colleagues can identify structural and instructional ambiguities, while target population members may reveal problems of comprehension, language, emotional reaction, and completion time that colleagues may not anticipate. A sample of 10 to 30 is commonly cited, though fewer may suffice if the tool is straightforward. All revisions made following the pilot should be documented and reported in the methods section as part of the methodological record (Table 10).⁵

THE DATA DICTIONARY

A data dictionary is a reference document that defines every variable in the dataset before data entry begins. At minimum it should record, for each variable: the variable name as it appears in the spreadsheet, a plain-language label describing what it represents, the scale or section it belongs to, the permissible range of values, the meaning of each numeric code, and the codes used for missing or non-applicable responses. It should also record any scoring or recoding rules applied after entry. The data dictionary allows a collaborating statistician, a co-investigator, or the researcher themselves at later date to understand the dataset without ambiguity. It should be maintained by the principal investigator, updated whenever a variable definition or coding decision changes, and stored alongside the dataset. A well-maintained data dictionary also serves as an audit trail for methodological decisions made during the study.

DATA SECURITY AND CONFIDENTIALITY

Participant data must be protected from the pint of collection onwards, which is standard ethical requirement. 11-13 participants should be identified using id numbers only, with a separate secured linkage log maintained apart from the data. For anonymous studies, no identifying information should appear on the tool itself, and this should be stated explicitly in the information sheet. Physical copies should be stored in a locked cabinet restricted to the research team, while digital data should be password-protected and access restricted to those directly involved in the study.

CONCLUSION

The design of a data collection tool is a methodological decision with direct consequences for data quality, participant experience, analytic validity, and the scientific contribution of the study. Postgraduate trainees who invest time in careful tool design - attending to sequence, format, coding, and pilot testing - are far less likely to encounter incomplete datasets, ambiguous variables, or unanalysable responses after recruitment has closed. The principles outlined in this guide apply across the range of study designs encountered in psychiatric research and provide a foundation for developing this skill systematically across a research career.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: Not required

REFERENCES

1. 1. Ohrbach R. Guidelines for establishing cultural equivalency of instruments. Available from: https://www.researchgate.net/publication/265229959_Guidelines_for_Establishing_Cultural_Equivalency_of_Instruments/citations. Accessed on 20 March 2026.
2. 2. Desai S, Patel N. ABC of face validity for questionnaire. Int J Pharm Sci Rev Res. 2020;65(1):164-8.
3. 3. Yusoff MSB. ABC of content validation and content validity index calculation. Education in Medicine Journal. 2019;11(2):49-54.
4. 4. Sharma H. How short or long should be a questionnaire for any research? Researchers dilemma in deciding the appropriate questionnaire length. Saudi J Anaesth. 2022;16(1):65-68.
5. 5. Dharaiya D. A practical primer on pilot study for postgraduate psychiatry students. Ann Indian Psychiatry. 2026.
6. 6. Dharaiya D. Online data collection in psychiatric research: a practical guide to platform selection, form design, and ethical conduct for postgraduate psychiatry students. Ind J Priv Psychiatry. 2026.
7. 7. Andrade C. Rating patients in different languages: reliability and validity. Indian J Psychol Med. 2024;46(3):268-9.
8. 8. Andrade C. A student's guide to the classification and operationalization of variables in the conceptualization and design of a clinical study: Part 2. Indian J Psychol Med. 2021;43(3):265-8.
9. 9. Andrade C. The young researcher's guide to creating data mastersheets. Indian J Psychol Med. 2022;44(6):618-9.
10. 10. Williams R. Coding missing data. University of Notre Dame Department of Sociology. Available from: https://academicweb.nd.edu/~rwilliam/stats3/MDCoding.pdf. Accessed on 20 March 2026.
11. 11. Ethical principles and practices for research involving human participants with mental illness. Psychiatr Serv. 2006;57(4):552-7.
12. 12. Indian Council of Medical Research. National Ethical Guidelines for Biomedical and Health Research Involving Human Participants [Internet]. New Delhi: ICMR; 2017 [cited 2026 April 21]. Available from: https://ethics.ncdirindia.org/asset/pdf/ICMR_National_Ethical_Guidelines.pdf. Accessed on 20 March 2026.
13. 13. Grover S, Sarkar S, Sahoo S. Ethics in psychiatric research. Indian J Psychiatry. 2022;64(Suppl 1):S68-80.