Design

The pilot study itself consists of both qualitative methods (a concurrent think aloud method and a semistructured interview) and quantitative methods (questionnaires). It compares the user experience of the two DiGAs, deprexis and Selfapy, that were the first—and by the time of planning the overall project, the only two permanent—approved DiGA by the BfArM for treating depressive disorders.

The screening study is planned as a cross-sectional trial and recruits geriatric patients with and without a diagnosis of depression. It includes a data collection path in a two-step approach. The first step includes the collection of data that is routinely collected during hospitalisation. If participants agree to participate in the second step, additional data on patients' media affinity is collected. Data of participants and reasons for non-participation in the further procedure will be analysed. In addition, the screening study is used to determine whether participants meet the eligible criteria for the pilot study and the subsequent RCT.

Recruitment in the pilot study

For the pilot study, recruitment took place from February to April 2024 in three departments of two hospitals: Psychiatry, Psychosomatics (both LWL University Hospital Bochum) and Geriatrics (Marien Hospital Herne), while the screening study took place exclusively in Geriatrics. Each hospital department contributes at least three participants for each of the two DiGAs, resulting in a sample size of at least 18 in total. With the calculated number of participants (nine for each DiGA), thematic saturation shall be achieved. Sample size will be adjusted, if thematic saturation is not reached. This is underlined by Virzi’s model that states that 80% of usability problems are recognised by four or five participants and that the most serious problems are already noticed by the first participant.23

Participants are recruited among all patients who are hospitalised during the recruitment period in the three hospital departments. The selection is criteria based. The pilot study aims at conducting a heterogeneous sample of inpatient participants in terms of their sociodemographic characteristics, health and digital literacy as well as media affinity. The first three patients included in the pilot study at each department start using the application Selfapy, while the next three patients included use the application deprexis.

In order to participate in the pilot study, the following inclusion criteria must be met: participant is 60 years or older, diagnosed with mild or moderate depressive disorder, owns a digital device (eg, smartphone or tablet) and gives written consent to take part in the study. A participant must be excluded if one of the following criteria occurs: participant shows acute psychotic symptoms, has acute suicidal tendencies or self-endangerment that results in a direct requirement for treatment, is severely intellectually impaired, has motor and sensory impairments that prevent using a digital device, has insufficient knowledge of German, has an advanced and incurable disease, is diagnosed with bipolar disorder or schizophrenia, participates currently in another study or is presently using a DiGA for treatment of depressive disorder. There is no distinction made in terms of gender. Decisive for the selection of the criteria were the need for participants to be stable enough for the study, fulfil the diagnosis the DiGAs were approved for and may receive inpatient follow-up care. In addition, it was organisationally easier to contact the participants if they were inpatients.

Recruitment in the screening study

The screening study is performed in the Department of Geriatric Medicine, including inpatient hospital care and outpatient day clinic for geriatric medicine, Marien Hospital Herne, representing the most vulnerable older patients. The study is designed to recruit potential participants for the pilot study or the subsequent RCT of the DiGA4Aged project in a cohort in whom it is anticipated that sufficient recruitment for the RCT may be difficult. Additionally, it collects further data and determines the extent to which geriatric hospital patients have experiences with and access to digital media. Therefore, all patients who are consecutively admitted in 2024 will be offered the opportunity to participate in the screening study. The Department of Geriatric Medicine treats around 2000 patients per year. Based on the experience of previous studies at this department and under the condition that participation in the study is optional while the inclusion criteria must be met, we expect a total number of 1800 participants.

The inclusion criterion is a minimum age of 60 years and older. Furthermore, patients must have an expected minimum stay of 14 days due to a formalised rehabilitation procedure during an acute care hospital stay, which comprises a minimum stay of 14 days.24 As a consequence, patients with a shorter length of stay will not be approached by the study team. In addition, the patient must be able to give informed consent. Patients who cannot participate due to the severity of their illness are excluded. No distinction is made in terms of gender.

All patients are listed in the hospital information system with their admission and planned discharge dates. The screening team operates independently and approaches the patients in person about the screening study within the last 3 days before their planned discharge. Patients are offered participation in the screening study in the described two-step approach. If they agree to participate in the more comprehensive branch, data will be collected by various questionnaires mentioned in the section Instruments. If patients do not wish to participate in the comprehensive screening study, they are asked for consent to use the routinely collected data. As the routinely collected data can be extracted from the hospital information system, the patient does not need to answer any further questions. In both cases, written informed consent is necessary for participation of the patient.

Procedure of the pilot study

The pilot study focuses on the usability of the DiGA Selfapy and deprexis. While Selfapy is approved in mild and moderate depressive disorder, deprexis covers the treatment of mild, moderate and severe depressive disorders. Selfapy can be used both as an app and a web application, whereas deprexis is solely offered as a web application. Both DiGAs are structured in several units to be worked on by users in individual sessions. The main function of those DiGAs is a communication-based interaction, in which users receive individualised information and recommendations based on their answers given within the units. Selfapy and deprexis both make use of behavioural therapy interventions and accordingly offer exercises for self-awareness and regulation. In addition, both DiGAs support the therapeutic work outside of the sessions by providing users with motivational e-mails (deprexis) and enabling digital contact to psychological counsellors if required (Selfapy). The recommended period of use in both cases is 90 days with a weekly usage time of 1–2 hours. Selfapy is available in German, while deprexis offers eight languages (Chinese, English, French, German, Greek, Italian, Portuguese, Swedish, Spanish).20

At the beginning of the pilot study, participants are informed about the procedure and agree to take part in the study by signing the consent form. Participants are free to choose whether the study is conducted with their own digital device (smartphone or tablet) or the provided study tablet. In both cases, the study conductors take care of accessing the web application and logging into the DiGA by using the registration data. In order to ensure comparability, the study conductors at each hospital department were equally trained with guidelines, developed for the pilot study. A sufficient number of test codes have been supplied by the DiGA developers in order to create test accounts prior to the meeting with the participant. This kind of access to the DiGA allows an identical use to the full version on prescription, except the feature by Selfapy of contacting a psychological counsellor via chat, which is not accessible via the test account.

The first part of the pilot study’s data collection involves four instruments. The participants complete a form for sociodemographic data and provide self-assessment of the following questionnaires: electronic health literacy by Revised German eHealth Literacy Scale (GR-eHEALS),25 health literacy by European Health Literacy Survey Questionnaire (HLS-EU-Q16)26 and media affinity including the frequency of use and knowledge of modern technology.27 A more detailed description of the data collection instruments can be found in the section Instruments.

After the completion of these four questionnaires, the second part of the pilot study using the concurrent think aloud method28 is initiated. The concurrent think aloud method is a common procedure in the field of digital healthcare for testing technologies with regard to usability, content and accuracy of fit.29 Users are asked to verbalise their thoughts while getting acquainted with the application for the first time.28 In addition, the participants receive a catalogue of tasks that are based on typical activities during initial use (eg, using the menu, moving through the various parts of the DiGA, open and view content). Different to verbalising thoughts and feelings after the usage, the hallmark of the concurrent think aloud method is verbalising all thoughts while performing the tasks.30 31 The use of the concurrent think aloud method allows for the capturing of users’ cognitive and emotional reactions and processes.32

This section of the procedure involves video recording with a camera placed behind the participant. It focuses on the tablet’s screen, captures the finger movements and commands performed by the participant. Simultaneously, the participant’s comments given during the task execution are recorded. The concurrent think aloud method begins with the start page of the DiGA after being logged in. The participants receive a sheet of tasks that slightly differ due to the character of each DiGA and are asked to perform those tasks while commenting on each step taken and verbalising emerging thoughts or problems during the completion. The study conductor accompanies the process mainly as an observer, giving a few reminders to verbalise thoughts if necessary. In addition, the study conductors are instructed to note any difficulties or peculiarities that arise during the course of action.

The third part obtains a semistructured interview to gather the patient’s individual impression of their initial use of the DiGA. The interview covers five clusters of topics with questions about their initial visual impression, accessibility of contents and an overall evaluation of the DiGA. In addition, the participants are asked about their impression of the research method, as the process of actively verbalising one’s thoughts is assumed to be unfamiliar for the participants. Semistructured interviews are conducted in order to record the participants’ individual experiences and impressions after their use of the DiGA. The basic framework for the interview structure is provided; however, additional or in-depth questions can be asked if more specific answers are needed. The guideline enables comparability between the interviews, while the flexibility of this interview form makes it possible to respond specifically to the participants’ individual impressions.33

With the completion of the interview, the video recording stops, and the final two questionnaires, Patient Health Questionnaire (PHQ-9)34 about severity of depression and System Usability Scale (SUS)35 about the perceived usability of the DiGA, are completed by the participant. Both questionnaires are described in the section Instruments.

The participants receive a small compensation for taking part in the pilot study. For the entire procedure of the pilot study, approximately 1.5 hours are set per participant. Directly afterwards, all gathered data is pseudonymised and secured by the study conductor.

The extraction and synthesis of the qualitative data will be carried out according to Rädiker and Kuckartz36 using the software MAXQDA, whereas quantitative data will be analysed by descriptive statistics.

The three parts of the data collection in the pilot study provide insight into the subjective and objective evaluation of the use of the DiGA. The evaluation will be based on the People at the Centre of Mobile Application Development usability model.37 The mixed-method approach enables to evaluate the user experience and usability of both DiGAs. User experience considers the user’s reactions and perceptions that arise before, during and after the usage.38 An important component of the user experience is usability, which is defined as the extent to which a system, product or service can be used by specific users in an explicit context of use to achieve defined goals effectively, efficiently and satisfactorily.39 By triangulating the data, it is possible to examine usability and user experience, which is crucial for assessing the extent to which the respective DiGA can be used by the target group. This is decisive for the RCT.

Procedure of the screening study

All consecutive geriatric patients aged 60 years and older undergo a comprehensive geriatric assessment within the first two working days after admission as part of the clinical routine. Some data of this initial assessment is used for the screening study later on, if participants provide consent. As patients are often severely impaired by their acute illness at the start of hospital stay, the additional study-related tasks are carried out within the last three working days before discharge.

If patients are willing to participate and give informed consent, the following data is extracted from the clinical routine data and the geriatric assessment: sociodemographic information, Barthel Index (BI)40 on admission and discharge, Clinical Frailty Scale (CFS),41 42 Confusion Assessment Method (CAM),43 Timed Up and Go (TUG) Test44 on admission and discharge, Montreal Cognitive Assessment (MoCA)45 on admission, the Depression in Old Age Scale (DIA-S)46 and main diagnosis and other study-relevant diagnoses.

Further study related data is collected at the patient’s bedside. The additional questioning includes the living situation at the time of admission, to what extent relatives are involved in care, retesting the MoCA and DIA-S on discharge, media affinity regarding frequency of use of modern technology and knowledge of modern technology,27 the GR-eHEALS,25 the willingness to use a DiGA and the PHQ-9,34 if the patients is eligible and willing to participate in the RCT. The PHQ-9 is only conducted if both questions on the behalf of the willingness to use DiGA in general and to support the treatment of depressive symptoms are answered positively, the DIA-S is above two points on discharge and there is no diagnosis of depressive disorder. The data collection of the screening study is collected in the web-based electronic data caption tool REDCap.47

The screening study enables both the recruitment of patients for the pilot study and subsequent RCT, as well as for the identification of clinical predictors for the non-use of digital media and a lack of willingness to use a DiGA. Descriptive statistics are planned to characterise the frequency and mode of media use. In addition, multiple linear and logistic regression will be used to determine predictors of media literacy and non-use of digital media in older patients. The results will allow to predict the use and non-use of digital media by geriatric hospital patients.

Possible predictors could be the availability of a private digital device and the presence of technical support by close relatives. Cognitive function assessed with MoCA and self-caring capacity assessed with BI are likewise highly interesting when analysing media use and participation in the RCT.

Instruments

Alongside the qualitative data, consisting of the concurrent think aloud method and the semistructured interview, a range of quantitative data is collected by various questionnaires. Figure 2 provides an overview of all instruments within the pilot and screening study.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Instruments of pilot and screening study.

Five questionnaires are used in both the pilot and the screening studies:

Sociodemographic data includes gender, age, care level, years of formal education, highest level of education and current occupation. Furthermore, pilot study collects the diagnosis of depressive disorder according to 10th revision of the International Classification of Diseases (ICD-10) as well as year of disease onset, number of previous treatments and whether antidepressive treatment is ongoing.

Media affinity is evaluated by two questionnaires regarding the frequency of use and knowledge of modern technology. They were used from a study on ageing experience and technology with kind permission of Schlomann et al at the University of Heidelberg.27 In the questionnaire on the frequency of use of modern technology, participants will be directed to focus on their active use of technical devices in the last 12 months. The technical devices include a stationary computer, laptop or notebook, smartphone, tablet, normal mobile phone without internet and use of internet in general. Answers are given for the self-reliant use of each device and include six levels from ‘never’ to ‘daily/almost daily’. The questionnaire on knowledge of modern technology measures to what extent the participants are familiar with the respective technical devices and applications. For each technical device (see above), patients should rate themselves with school grades, according to the German grading system, ranging from very good (grade one) to unsatisfactory (grade six).

The eHEALS was developed to measure the digital health literacy of consumers. It measures consumers’ existing knowledge, self-confidence and self-perceived competence in finding, evaluating and applying electronic health information to solve health problems. The eHEALS is based on consumers’ self-assessment and comprises eight statements that must be rated on a five-point Likert scale.48 The GR-eHEALS with 16 items is a validated instrument for measuring health literacy in German and addresses methodological weaknesses of the primary German version of eHEALS with regard to test development and validation.25

The PHQ-9 is a self-report version of the Primary Care Evaluation of Mental Disorders (PRIME-MD) diagnostic tool for common mental disorders.49 Each of the nine Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria for diagnosing depressive disorders is rated on a 4-level scale between ‘not at all’ and ‘almost every day’ in the PHQ-9. The sum score is graded as minimal, mild, moderately severe and severe depression.34

Two questionnaires are exclusively used within the pilot study:

Health literacy is associated as a relevant factor with maintaining one’s own health and coping with illness. It is measured using the short form of the established HLS-EU-Q in German. The short form uses 16 items to assess the four dimensions of health literacy (accessing, understanding, assessing and using health information) in the areas of disease prevention, health promotion and healthcare. The participants rate on a 4-level scale from ‘very easy’ to ‘very difficult’ how they perceive certain tasks or activities in connection with disease prevention, health promotion and healthcare. The 16 items are a balanced representation of the 47 items in the original version.26

The SUS is an established questionnaire for measuring the perceived usability of an application. It consists of 10 items that are rated on a scale from ‘completely disagree’ to ‘completely agree’. The German version of the SUS, provided by Digital Equipment Corporation, 1986,35 will be used in the pilot study.

The screening study uses the following data:

The BI assesses the observed ability to care for oneself over the last 2 days. The items include the abilities of eating, getting up and moving, cleaning oneself, using the toilet, bathing or showering, getting up and walking, climbing stairs, dressing and undressing, bowel and urinary incontinence. The rating is calculated by adding the ratings of the individual items. The score ranges from 0 to 100, with high scores indicating less impairment.40 43 In addition to recording the status of independence, the BI can also be used to monitor activities of daily life and therefore record changes during the rehabilitation process.40 50

The CFS was developed to quickly and reliably classify older people in terms of the severity of frailty. CFS is based on a global clinical judgement. This involves assigning people to one of nine classifications from very fit to terminally ill.41 42

A further survey instrument used is the CAM. It is suitable for screening and diagnosing delirium as it correlates with the DSM-V and ICD-10 criteria for delirium. The scale used is based on the four components of acute onset of symptoms, attention deficit, formal thought disorder and clouding of consciousness.43

The TUG Test provides an integrative assessment of mobility, balance, co-ordination, joint function and strength of the lower limb.44 As part of the TUG Test, the patient is asked to rise from a seated position without assistance from a chair with armrests, walk 3 m and back and sit down again. Assistive devices such as a rollator are permitted in this test. The test is evaluated using the time taken by the patient to complete the test.43

The MoCA is a tool to detect any degree of cognitive impairment.45 51 It includes various components that focus on short-term memory, language ability, attention, visuoconstructive abilities, orientation and executive functions.45 The various components are awarded points, with the best and maximum total score being 30.43

The DIA-S is a screening instrument for self-assessment and consists of 10 items that are answered dichotomised with yes or no. The items refer to the state of mind during the last 14 days. The DIA-S is comparable to other screening instruments for depressive disorders but is better suited for the use in the geriatric setting due to its practicality.43 46

Lastly all participants are asked about their willingness to use a DiGA in general and regardless of illness. A further question relates to the willingness of using a DiGA to support the treatment of depressive symptoms.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.