Advertisement

Predictive models for response to non-invasive brain stimulation in stroke: A critical review of opportunities and pitfalls

  • Author Footnotes
    1 Authors contributed equally.
    Maximilian J. Wessel
    Footnotes
    1 Authors contributed equally.
    Affiliations
    Defitech Chair of Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology Lausanne (EPFL), Geneva, Switzerland

    Defitech Chair of Clinical Neuroengineering, Clinique Romande de Réadaptation, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL Valais), Sion, Switzerland
    Search for articles by this author
  • Author Footnotes
    1 Authors contributed equally.
    Philip Egger
    Footnotes
    1 Authors contributed equally.
    Affiliations
    Defitech Chair of Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology Lausanne (EPFL), Geneva, Switzerland

    Defitech Chair of Clinical Neuroengineering, Clinique Romande de Réadaptation, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL Valais), Sion, Switzerland
    Search for articles by this author
  • Friedhelm C. Hummel
    Correspondence
    Corresponding author. Defitech Chair of Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute, Swiss Federal Institute of Technology (EPFL), Campus Biotech, Chemin des Mines 9, 1202 Geneva, Switzerland.
    Affiliations
    Defitech Chair of Clinical Neuroengineering, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology Lausanne (EPFL), Geneva, Switzerland

    Defitech Chair of Clinical Neuroengineering, Clinique Romande de Réadaptation, Center for Neuroprosthetics (CNP) and Brain Mind Institute (BMI), Swiss Federal Institute of Technology (EPFL Valais), Sion, Switzerland

    Clinical Neuroscience, University of Geneva Medical School, Geneva, Switzerland
    Search for articles by this author
  • Author Footnotes
    1 Authors contributed equally.
Open AccessPublished:September 21, 2021DOI:https://doi.org/10.1016/j.brs.2021.09.006

      Highlights

      • Manuscript: “Predictive models for response to non-invasive brain stimulation in stroke: a critical review of opportunities and pitfalls”
      • Need for personalized NIBS application to improve response rates and effect size of individual patients
      • Allocation of personalized NIBS relies on robust predictive models
      • Most work on NIBS response-determining factors is based on in-sample associations
      • Associations within a sample are not valid for making generalizable predictions, internal and external validation needed for generalizability
      • Large and unbiased data sets are critical to establish good predictive models towards personalized NIBS

      Abstract

      Background

      Noninvasive brain stimulation has been successfully applied to improve stroke-related impairments in different behavioral domains. Yet, clinical translation is limited by heterogenous outcomes within and across studies. It has been proposed to develop and apply noninvasive brain stimulation in a patient-tailored, precision medicine-guided fashion to maximize response rates and effect magnitude. An important prerequisite for this task is the ability to accurately predict the expected response of the individual patient.

      Objective

      This review aims to discuss current approaches studying noninvasive brain stimulation in stroke and challenges associated with the development of predictive models of responsiveness to noninvasive brain stimulation.

      Methods

      Narrative review.

      Results

      Currently, the field largely relies on in-sample associational studies to assess the impact of different influencing factors. However, the associational approach is not valid for making claims of prediction, which generalize out-of-sample. We will discuss crucial requirements for valid predictive modeling in particular the presence of sufficiently large sample sizes.

      Conclusion

      Modern predictive models are powerful tools that must be wielded with great care. Open science, including data sharing across research units to obtain sufficiently large and unbiased samples, could provide a solid framework for addressing the task of building robust predictive models for noninvasive brain stimulation responsiveness.

      Keywords

      1. Introduction

      Stroke is a major cause of disability worldwide and its overall burden has increased in recent years [
      • Feigin V.L.
      • Norrving B.
      • Mensah G.A.
      Global burden of stroke.
      ]. Patients are often significantly limited by the functional consequences of their stroke. For instance, roughly 65% of stroke survivors cannot incorporate their affected hand into their activities of daily living six months post-stroke, roughly 50% have cognitive impairments in their chronic phase, and only around 25% return to their full pre-stroke level of everyday participation and physical functioning [
      • Dobkin B.H.
      Clinical practice. Rehabilitation after stroke.
      ,
      • Kapoor A.
      • Lanctôt K.L.
      • Bayley M.
      • Kiss A.
      • Herrmann N.
      • Murray B.J.
      • et al.
      “Good outcome” isn't good enough: cognitive impairment, depressive symptoms, and social restrictions in physically recovered stroke patients.
      ]. This highlights the great necessity of developing novel therapeutic strategies to promote stroke rehabilitation. Several interventions aiming at promoting underlying brain plasticity are currently under investigation [
      • Raffin E.
      • Hummel F.C.
      Restoring motor functions after stroke: multiple approaches and opportunities.
      ]. In this review, we will discuss non-invasive brain stimulation (NIBS) techniques, provide an overview of the current applications, and discuss possible future steps towards precision medicine and the prerequisites for accomplishing this task.
      Currently, the most widely used NIBS techniques are transcranial direct current simulation (tDCS) and repetitive transcranial magnetic stimulation (rTMS), for details please see panel 1. In stroke rehabilitation research, NIBS mostly aims at (i) rectifying unhealthy brain states such as maladaptive disbalanced interhemispheric inhibition or (ii) reinforcing effects of training-based interventions [
      • Hummel F.C.
      • Cohen L.G.
      Non-invasive brain stimulation: a new strategy to improve neurorehabilitation after stroke?.
      ,
      • Wessel M.J.
      • Zimerman M.
      • Hummel F.C.
      Non-invasive brain stimulation: an interventional tool for enhancing behavioral training after stroke.
      ]. Different stroke-related target symptoms have been addressed. Motor stroke, aphasia, and hemispatial neglect are common areas of application [
      • Lefaucheur J.-P.
      • André-Obadia N.
      • Antal A.
      • Ayache S.S.
      • Baeken C.
      • Benninger D.H.
      • et al.
      Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS).
      ]. For example, initial research studying anodal (excitatory) tDCS applied to the ipsilesional primary motor cortex (M1) in chronic motor stroke patients documented positive effects on paretic hand function [
      • Hummel F.
      • Celnik P.
      • Giraux P.
      • Floel A.
      • Wu W.-H.
      • Gerloff C.
      • et al.
      Effects of non-invasive cortical stimulation on skilled motor function in chronic stroke.
      ,
      • Hummel F.C.
      • Voller B.
      • Celnik P.
      • Floel A.
      • Giraux P.
      • Gerloff C.
      • et al.
      Effects of brain polarization on reaction times and pinch force in chronic stroke.
      ]. Similarly, proof-of-principle rTMS-research has shown beneficial effects on motor function and stroke-related impairment scales [
      • Khedr E.M.
      • Ahmed M.A.
      • Fathy N.
      • Rothwell J.C.
      Therapeutic trial of repetitive transcranial magnetic stimulation after acute ischemic stroke.
      ,
      • Mansur C.G.
      • Fregni F.
      • Boggio P.S.
      • Riberto M.
      • Gallucci-Neto J.
      • Santos C.M.
      • et al.
      A sham stimulation-controlled trial of rTMS of the unaffected hemisphere in stroke patients.
      ]. This initial work was conducted about 15 years ago and has been partially replicated and extended in following years, for instance by combining NIBS with behavioral training paradigms [
      • Zimerman M.
      • Heise K.F.
      • Hoppe J.
      • Cohen L.G.
      • Gerloff C.
      • Hummel F.C.
      Modulation of training by single-session transcranial direct current stimulation to the intact motor cortex enhances motor skill acquisition of the paretic hand.
      ] or other technologies such as robot-based interventions [
      • Hesse S.
      • Waldner A.
      • Mehrholz J.
      • Tomelleri C.
      • Pohl M.
      • Werner C.
      Combined transcranial direct current stimulation and robot-assisted arm training in subacute stroke patients: an exploratory, randomized multicenter trial.
      ]. However, the further clinical translation of the approach is limited so far. Based on current guidelines, strongest level A evidence (definite efficacy) has only been reported for contralesional low-frequency M1 rTMS targeting motor impairments in the subacute phase [
      • Lefaucheur J.-P.
      • Aleman A.
      • Baeken C.
      • Benninger D.H.
      • Brunelin J.
      • Di Lazzaro V.
      • et al.
      Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): an update (2014-2018).
      ]. The application of ipsilesional high-frequency M1 rTMS for enhancing motor function in the post-acute phase and low-frequency rTMS of the right inferior frontal gyrus targeting non-fluent aphasia in the chronic phase has reached level B evidence (probable efficacy) [
      • Lefaucheur J.-P.
      • Aleman A.
      • Baeken C.
      • Benninger D.H.
      • Brunelin J.
      • Di Lazzaro V.
      • et al.
      Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): an update (2014-2018).
      ]. For the tDCS technique, there is currently not sufficient data available to reach a solid recommendation on possible therapeutic efficacy for motor stroke or aphasia [
      • Lefaucheur J.-P.
      • Antal A.
      • Ayache S.S.
      • Benninger D.H.
      • Brunelin J.
      • Cogiamanian F.
      • et al.
      Evidence-based guidelines on the therapeutic use of transcranial direct current stimulation (tDCS).
      ].
      One possible cause for the limited clinical translation of the discussed NIBS approaches to date is the emergence of distinct responder and non-responder patterns. Examples of this bifurcation include lower responsiveness to rTMS protocols in patients with cortical lesions [
      • Lefaucheur J.-P.
      • Aleman A.
      • Baeken C.
      • Benninger D.H.
      • Brunelin J.
      • Di Lazzaro V.
      • et al.
      Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): an update (2014-2018).
      ] or the substantial variation in outcomes across tDCS studies investigating effects on upper extremity function [
      • Elsner B.
      • Kugler J.
      • Pohl M.
      • Mehrholz J.
      Transcranial direct current stimulation (tDCS) for improving activities of daily living, and physical and cognitive functioning, in people after stroke.
      ]. It has been proposed that the heterogeneity in response rates might be reduced by tailoring the NIBS protocols to the phenomenological subgroup (precision medicine) or individual patients (personalized medicine) [
      • Cunningham D.A.
      • Potter-Baker K.A.
      • Knutson J.S.
      • Sankarasubramanian V.
      • Machado A.G.
      • Plow E.B.
      Tailoring brain stimulation to the nature of rehabilitative therapies in stroke: a conceptual framework based on their unique mechanisms of recovery.
      ,
      • Koch P.J.
      • Hummel F.C.
      Toward precision medicine: tailoring interventional strategies based on noninvasive brain stimulation for motor recovery after stroke.
      ,
      • Leuchter A.F.
      • Corlier J.
      A precision medicine approach to repetitive Transcranial Magnetic Stimulation (rTMS).
      ]. Yet, these approaches have not been convincingly tested so far.
      We argue that there are two critical prerequisites for testing a precision medicine approach. First, a sufficient number of effective stimulation protocols must be available. In this regard, research has extended the spatial parameter space by investigating alternative stimulation targets, such as the cerebellum [
      • Wessel M.J.
      • Hummel F.C.
      Non-invasive cerebellar stimulation: a promising approach for stroke recovery?.
      ,
      • Koch G.
      • Bonnì S.
      • Casula E.P.
      • Iosa M.
      • Paolucci S.
      • Pellicciari M.C.
      • et al.
      Effect of cerebellar stimulation on gait and balance recovery in patients with hemiparetic stroke: a randomized clinical trial.
      ], the premotor cortex [
      • Plow E.B.
      • Cunningham D.A.
      • Varnerin N.
      • Machado A.
      Rethinking stimulation of the brain in stroke rehabilitation: why higher motor areas might be better alternatives for patients with greater impairments.
      ], or contralesional parietal areas [
      • Cunningham D.A.
      • Potter-Baker K.A.
      • Knutson J.S.
      • Sankarasubramanian V.
      • Machado A.G.
      • Plow E.B.
      Tailoring brain stimulation to the nature of rehabilitative therapies in stroke: a conceptual framework based on their unique mechanisms of recovery.
      ,
      • Cunningham D.A.
      • Varnerin N.
      • Machado A.
      • Bonnett C.
      • Janini D.
      • Roelle S.
      • et al.
      Stimulation targeting higher motor areas in stroke rehabilitation: a proof-of-concept, randomized, double-blinded placebo-controlled study of effectiveness and underlying mechanisms.
      ]. Furthermore, recent technical developments such as of state-dependent electroencephalography-triggered transcranial magnetic stimulation (TMS) have enlarged the temporal parameter space [
      • Zrenner C.
      • Desideri D.
      • Belardinelli P.
      • Ziemann U.
      Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex.
      ].
      In this review, we focus on the second prerequisite – the development of robust models for predicting response rates for the respective NIBS interventions. We will summarize and discuss current approaches, which largely rely on in-sample statistical associations (data mining). These approaches are valid for gaining insights about potential underlying mechanisms or for identifying possible candidate predictor variables for further testing. However, it is critical to note that in-sample statistical associations alone are insufficient evidence for claims of out-of-sample predictive power [
      • Poldrack R.A.
      • Huckins G.
      • Varoquaux G.
      Establishment of best practices for evidence for prediction: a review.
      ]. If the field continues to rely on data mining/associational approaches, it will be unable to identify robust biomarkers for predicting NIBS responsiveness. However, a shift towards the appropriate use of predictive modeling techniques may overcome this limitation. In the second section of the review, we will discuss the basics of predictive modeling approaches, providing a brief overview specifically targeted to an audience of applied translational neuroscientists and clinicians. Furthermore, we highlight critical prerequisites and limitations of predictive modeling techniques. Hereby, we strive to inform future study designs assessing predictors/biomarkers of NIBS responsiveness. If successful, this could be an important step towards developing NIBS protocols suitable for precision medicine.

      2. Part 1: current status of associational approaches

      Initial research has established in-sample associations between parameters derived from different domains spanning from simple behavioral to more complex electrophysiological or imaging-based metrics and NIBS responsiveness. Some important exemplary studies are discussed below, for a detailed overview please see Table 1. It is tempting to overgeneralize these findings and make interferences beyond the studied sample. However, this approach has several pitfalls, as discussed in detail in the section on predictive modeling, and should be avoided. Yet, in-sample associations can be useful to generate insights into potential underlying mechanisms or for identifying candidate predictors to be tested in future studies.
      Table 1Overview of identified associational studies relating parameters derived from different assessment domains with response to NIBS. ∗: 10–20 EEG positions F3-anode, F3-cathode, F4-anode, or F4- cathode to return electrode at contralateral mastoid; ∗∗: with non-parametric permutation testing; ∗∗∗: stepwise backward selection; AI: activity index scale (disability); ANOVA: analysis of variance; ARAT: Action Research Arm Test; BCI: brain-computer interface; cM1: contralesional M1; CBS: Catherine Bergego Scale; COR: correlation analysis (unspecified); CORpa: partial correlations; CORpe: Pearson Product-Moment Correlation; CORsp: Spearman's Rank-Order Correlation; CORspm: correlation analysis, behavioral covariate was added to SPM design matrix; cPPC: contralesional, left posterior parietal cortex; CST: corticospinal tract; cTBS: continuous theta burst stimulation; DCM: dynamic causal modeling; DD: directional diffusivity; DTI: diffusion tensor imaging; ERSα: event-related synchronization in α-band; FA: fractional anisotropy; FMA-UE: Fugl-Meyer Assessment for upper extremity; fMRI: functional magnetic resonance imaging; fNIRS: functional near-infrared spectroscopy; FT: maximum index finger tapping frequency; GABA: γ-amino butyric acid; GF: maximum grip force; iM: ipsilesional M1; iSP ratio: ipsilateral silent period (contralesional/ipsilesional); iTBS: intermittent theta-burst stimulation; JTT: Jebsen Taylor hand function test; LF: low frequency; M1-M1: bihemispheric montage; MEG: magnetoencephalography; MEP: motor evoked potential; MFT: manual function test; MLR: multiple linear regression; MULR: multinomial logistic regression; MRI: magnetic resonance imaging; MRS: magnetic resonance spectroscopy; MVRA: multivariate regression analysis; MWUT: Mann-Whitney U test; Oz: occipital zero position of EEG 10–20 system; PRI: power ratio index (δ+θ)/(α+β); RT: reaction time; rTMS: repetitive transcranial magnetic stimulation; S1: primary somatosensory cortex; SICI: short intracortical inhibition; SLR: simple linear regression; struct: structural MRI; TCT: thalamocortical tract; tDCS: transcranial direct current stimulation; TMS: transcranial magnetic stimulation; VAS: visual analog scale for self-assessment of the level pain; VLSM; voxel-based lesion-symptom mapping.
      AuthorYearRef.NIBS protocolDomainCohortResponsiveness metricPredictorStatisticsCross-vali-dationMain finding
      Goto et al.2008[
      • Goto T.
      • Saitoh Y.
      • Hashimoto N.
      • Hirata M.
      • Kishima H.
      • Oshino S.
      • et al.
      Diffusion tensor fiber tracking in patients with central post-stroke pain; correlation with efficacy of repetitive transcranial magnetic stimulation.
      ]
      5 Hz rTMS M1PainChronic stroke (N = 13)VAS score decrease >30% (in relation to baseline)MRI - DWIMWUTnoHigher ratio affected/unaffected side for CST and TCT in rTMS-effective group
      Nowak et al.2008[
      • Nowak D.A.
      • Grefkes C.
      • Dafotakis M.
      • Eickhoff S.
      • Küst J.
      • Karbe H.
      • et al.
      Effects of low-frequency repetitive transcranial magnetic stimulation of the contralesional primary motor cortex on movement kinematics and neural activity in subcortical stroke.
      ]
      1 Hz rTMS cM1MotorSubacute stroke (N = 15)Percentage of improvement in finger tapping frequency (in relation to control stimulation)MRI - fMRICORspmnoOveractivity (higher) of the contralesional dorsal premotor cortex, contralesional parietal operculum, and ipsilesional mesial frontal cortex at baseline was associated with the level of improvement (more) in hand motor function after the intervention
      Ameli et al.2009[
      • Ameli M.
      • Grefkes C.
      • Kemper F.
      • Riegg F.P.
      • Rehme A.K.
      • Karbe H.
      • et al.
      Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
      ]
      10 Hz rTMS iM1MotorAcute to chronic stroke (N = 29)Median split of rTMS-induced percentage change to baseline (finger tapping frequency)(i) MRI - structANOVAnoResponders to the intervention had a subcortical lesion pattern
      Ameli et al.2009[
      • Ameli M.
      • Grefkes C.
      • Kemper F.
      • Riegg F.P.
      • Rehme A.K.
      • Karbe H.
      • et al.
      Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
      ]
      10 Hz rTMS iM1MotorAcute to chronic stroke (N = 18)Percentage change to baseline (finger tapping frequency)(ii) MRI - fMRICORspnoNeural activity at baseline within iM1 was positively associated with rTMS response
      Emara et al.2009[
      • Emara T.
      • El Nahas N.
      • Elkader H.A.
      • Ashour S.
      • El Etrebi A.
      MRI can predict the response to therapeutic repetitive transcranial magnetic stimulation (rTMS) in stroke patients.
      ]
      5 Hz rTMS iM1 or 1 Hz rTMS cM1MotorSubacute stroke (N = 60)Differences in AI scale across subgroupsMRI - structANOVAnoa. Lower responsiveness for patients with total anterior stroke, b. Lower responsiveness for patients with cortical stroke
      Halko et al.2011[
      • Halko M.A.
      • Datta A.
      • Plow E.B.
      • Scaturro J.
      • Bikson M.
      • Merabet L.B.
      Neuroplastic changes following rehabilitative training correlate with regional electrical field induced with tDCS.
      ]
      atDCS OzVisualCase study chronic stroke (N = 1)Change in fMRI signal to baselinefMRI + computa-tional modelingCORnoRelationship between changes in fMRI signal after visual rehabilitation and electric field map
      Lindenberg et al.2012[
      • Lindenberg R.
      • Zhu L.L.
      • Rüber T.
      • Schlaug G.
      Predicting functional motor potential in chronic stroke patients using diffusion tensor imaging.
      ]
      tDCS M1-M1MotorChronic stroke (N = 15)Proportional change in WMFT score (with respect to baseline)MRI - DTICORpanoGreater gains in motor function were related to higher FA values and lower DD values of transcallosal and ipsilesional cortico-spinal tracts
      Zimerman et al.2012[
      • Zimerman M.
      • Heise K.F.
      • Hoppe J.
      • Cohen L.G.
      • Gerloff C.
      • Hummel F.C.
      Modulation of training by single-session transcranial direct current stimulation to the intact motor cortex enhances motor skill acquisition of the paretic hand.
      ]
      ctDCS cM1MotorChronic stroke (N = 12)Behavioral online improvement (ratio) in tDCS conditionTMS - SICISLRnoRelationship between tDCS-induced improvement (larger) during training and the tDCS-induced modulation of SICI (stronger)
      O'Shea et al.2014[
      • O'Shea J.
      • Boudrias M.-H.
      • Stagg C.J.
      • Bachtiar V.
      • Kischka U.
      • Ju Blicher
      • et al.
      Predicting behavioural response to TDCS in chronic motor stroke.
      ]
      (i) atDCS iM1MotorChronic stroke (N = 8)Percentage change (delta) in RT contrasting real tDCS to shamMRI - MRSCORpenoRelationship between ipsilesional M1 GABA levels (higher) and responsiveness (larger) to atDCS when compared to sham
      O'Shea et al.2014[
      • O'Shea J.
      • Boudrias M.-H.
      • Stagg C.J.
      • Bachtiar V.
      • Kischka U.
      • Ju Blicher
      • et al.
      Predicting behavioural response to TDCS in chronic motor stroke.
      ]
      (ii) ctDCS cM1MotorChronic stroke (N = 13)Clinical - time since stroke + FMA-UEMLRnoThe longer the time since stroke and the higher (better) the FMA-UE the greater the ctDCS-associated gain
      Brodie et al.2014[
      • Brodie S.M.
      • Borich M.R.
      • Boyd L.A.
      Impact of 5-Hz rTMS over the primary sensory cortex is related to white matter volume in individuals with chronic stroke.
      ]
      5 Hz rTMS iM1MotorChronic stroke (N = 22)Absolute postinterventional change in response time (with respect to baseline)MRI - structCORpe/MLRnoPositive association between white matter volume in S1 and behavioral change in the interventional group
      Lai et al.2015[
      • Lai C.-J.
      • Wang C.-P.
      • Tsai P.-Y.
      • Chan R.-C.
      • Lin S.-H.
      • Lin F.-G.
      • et al.
      Corticospinal integrity and motor impairment predict outcomes after excitatory repetitive transcranial magnetic stimulation: a preliminary study.
      ]
      iTBS iM1MotorSubacute and chronic stroke (N = 72)Postinterventional WMFT(i) Clinical - motor function at baselineCORspnoBetter motor function was associated with better iTBS response
      Lai et al.2015[
      • Lai C.-J.
      • Wang C.-P.
      • Tsai P.-Y.
      • Chan R.-C.
      • Lin S.-H.
      • Lin F.-G.
      • et al.
      Corticospinal integrity and motor impairment predict outcomes after excitatory repetitive transcranial magnetic stimulation: a preliminary study.
      ]
      iTBS iM1MotorSubacute and chronic stroke (N = 72)Change (delta) in WMFT, RT, finger tapping frequency to baseline(ii) TMS – MEP at baselineCORspnoHigher MEPs were associated with better iTBS response
      Lee et al.2015[
      • Lee J.H.
      • Kim S.B.
      • Lee K.W.
      • Kim M.A.
      • Lee S.J.
      • Choi S.J.
      Factors associated with upper extremity motor recovery after repetitive transcranial magnetic stimulation in stroke patients.
      ]
      10 Hz rTMS iM1MotorSubacute stroke (N = 29)Classification in responder and non-responder subgroups based on the minimal clinically important difference of MFT (with respect to baseline)TMS - MEPMVRAnoAssociation with presence of MEP response and larger improvement after the intervention
      Diekhoff-Krebs et al.2017[
      • Diekhoff-Krebs S.
      • Pool E.-M.
      • Sarfeld A.-S.
      • Rehme A.K.
      • Eickhoff S.B.
      • Fink G.R.
      • et al.
      Interindividual differences in motor network connectivity and behavioral response to iTBS in stroke patients.
      ]
      iTBS iM1MotorChronic stroke (N = 14)Composite motor improvement score (including: GF, FT, JTT, referenced to: control TBS and baseline)MRI - fMRI (DCM)CORpenoHigher coupling strength between iSMA and iM1 and stronger inhibitory effect from iM1 to cM1 at baseline were associated with larger TBS response
      Norise et al.2017[
      • Norise C.
      • Sacchetti D.
      • Hamilton R.
      Transcranial direct current stimulation in post-stroke chronic aphasia: the impact of baseline severity and task specificity in a pilot sample.
      ]
      Indivi-dualized tDCS to frontal lobe∗Lang-uageChronic stroke (N = 9)Proportion change from baselineClinical -baseline severityCORspnoBaseline language ability (worse) was associated with improvement (larger) after active tDCS
      Pavlova et al.2017[
      • Pavlova E.L.
      • Lindberg P.
      • Khan A.
      • Ruschkowski S.
      • Nitsche M.A.
      • Borg J.
      Transcranial direct current stimulation combined with visuo-motor training as treatment for chronic stroke patients.
      ]
      atDCS M1MotorChronic stroke (N = 11)tDCS-induced change (delta) in FMA motor scoreBehavioral - mean hold forceSLRnoMean hold force during the first treatment session was associated with change of FMA-UE
      Mane et al.2019[
      • Mane R.
      • Chew E.
      • Phua K.S.
      • Ang K.K.
      • Robinson N.
      • Vinod A.P.
      • et al.
      Prognostic and monitory EEG-biomarkers for BCI upper-limb stroke rehabilitation.
      ]
      tDCS M1-M1MotorChronic stroke (N = 10)tDCS-BCI-induced change (delta) in FMA-UE (with respect to baseline)EEG - PRICORsp∗∗noPRI was best predictor for gains in FMA-UE after a tDCS-BCI intervention
      Nyffeler et al.2019[
      • Nyffeler T.
      • Vanbellingen T.
      • Kaufmann B.C.
      • Pflugshaupt T.
      • Bauer D.
      • Frey J.
      • et al.
      Theta burst stimulation in neglect after stroke: functional outcome and response variability origins.
      ]
      cTBS cPPCNeglectSubacute stroke (N = 30)Hierarchical cluster analysis of delta CBS scores (discharge to admission)MRI - structVLSMnocTBS responders had intact interhemispheric connections within the corpus callosum
      Tamashiro et al.2019[
      • Tamashiro H.
      • Kinoshita S.
      • Okamoto T.
      • Urushidani N.
      • Abo M.
      Effect of baseline brain activity on response to low-frequency rTMS/intensive occupational therapy in poststroke patients with upper limb hemiparesis: a near-infrared spectroscopy study.
      ]
      1 Hz rTMS cM1MotorChronic stroke (N = 59)rTMS-induced change (delta) in FMA-UE and WMFT (with respect to baseline)fNIRSCORspnoPositive association between the dominance of the unaffected hemisphere and the response to the rTMS protocol
      Baltar et al.2020[
      • Baltar A.
      • Piscitelli D.
      • Marques D.
      • Shirahige L.
      • Monte-Silva K.
      Baseline motor impairment predicts transcranial direct current stimulation combined with physical therapy-induced improvement in individuals with chronic stroke.
      ]
      atDCS iM1 or ctDCS cM1 or tDCS M1-M1MotorChronic stroke (N = 80)Post-treatment FMA-UEClinical - FMA-UEMLR∗∗∗noFMA-UE was a significant predictor of tDCS responsiveness - higher baseline FMA-UE more responsiveness
      Hamaguchi et al.2020[
      • Hamaguchi T.
      • Yamada N.
      • Hada T.
      • Abo M.
      Prediction of motor recovery in the upper extremity for repetitive transcranial magnetic stimulation and occupational therapy goal setting in patients with chronic stroke: a retrospective analysis of prospectively collected data.
      ]
      1 Hz rTMS cM1MotorLate subacute and chronic stroke (N = 1254)Treatment induced change (delta) in FMA-UE (with respect to baseline), literature-based stratification in 3 groups of responsivenessClinical - FMA-UEMULRnoResponder pattern was more frequent in patients with stronger impairment (FMA-UE) at baseline
      Kuo et al.2020[
      • Kuo I.-J.
      • Tang C.-W.
      • Tsai Y.-A.
      • Tang S.-C.
      • Lin C.-J.
      • Hsu S.-P.
      • et al.
      Neurophysiological signatures of hand motor response to dual-transcranial direct current stimulation in subacute stroke: a TMS and MEG study.
      ]
      tDCS M1-M1MotorSubacute stroke (N = 18)MEP amplitudes normalized to baseline and sham(i) TMS - iSPMLR∗∗∗noRelationship between iSP ratio (greater) and tDCS-induced MEP increase (smaller)
      Kuo et al.2020[
      • Kuo I.-J.
      • Tang C.-W.
      • Tsai Y.-A.
      • Tang S.-C.
      • Lin C.-J.
      • Hsu S.-P.
      • et al.
      Neurophysiological signatures of hand motor response to dual-transcranial direct current stimulation in subacute stroke: a TMS and MEG study.
      ]
      tDCS M1-M1MotorSubacute stroke (N = 18)Absolute FMA-UE and ARAT scores(ii) MEG - ERSαCORnotDCS-induced increase (larger) in ERSα was associated with FMA-UE and ARAT (higher)
      It is of note that the procedures for determining a responsiveness metric differ across studies and there is no unique and universally accepted way. Some examples are computing absolute (delta) or relative changes (percentage, ratio) to control stimulation (e.g. Ref. [
      • Nowak D.A.
      • Grefkes C.
      • Dafotakis M.
      • Eickhoff S.
      • Küst J.
      • Karbe H.
      • et al.
      Effects of low-frequency repetitive transcranial magnetic stimulation of the contralesional primary motor cortex on movement kinematics and neural activity in subcortical stroke.
      ]) or to baseline (e.g. Refs. [
      • Ameli M.
      • Grefkes C.
      • Kemper F.
      • Riegg F.P.
      • Rehme A.K.
      • Karbe H.
      • et al.
      Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
      ,
      • Hamaguchi T.
      • Yamada N.
      • Hada T.
      • Abo M.
      Prediction of motor recovery in the upper extremity for repetitive transcranial magnetic stimulation and occupational therapy goal setting in patients with chronic stroke: a retrospective analysis of prospectively collected data.
      ]), reporting of proportional change (e.g. Ref. [
      • Lindenberg R.
      • Zhu L.L.
      • Rüber T.
      • Schlaug G.
      Predicting functional motor potential in chronic stroke patients using diffusion tensor imaging.
      ]), applying cut-off-based procedures (median split e.g. Ref. [
      • Ameli M.
      • Grefkes C.
      • Kemper F.
      • Riegg F.P.
      • Rehme A.K.
      • Karbe H.
      • et al.
      Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
      ], splitting based on the minimal clinically important difference e.g. Ref. [
      • Lee J.H.
      • Kim S.B.
      • Lee K.W.
      • Kim M.A.
      • Lee S.J.
      • Choi S.J.
      Factors associated with upper extremity motor recovery after repetitive transcranial magnetic stimulation in stroke patients.
      ]) towards more complex approaches such as the use of composite scores (e.g. Ref. [
      • Diekhoff-Krebs S.
      • Pool E.-M.
      • Sarfeld A.-S.
      • Rehme A.K.
      • Eickhoff S.B.
      • Fink G.R.
      • et al.
      Interindividual differences in motor network connectivity and behavioral response to iTBS in stroke patients.
      ]), or hierarchical cluster analysis (e.g. Ref. [
      • Nyffeler T.
      • Vanbellingen T.
      • Kaufmann B.C.
      • Pflugshaupt T.
      • Bauer D.
      • Frey J.
      • et al.
      Theta burst stimulation in neglect after stroke: functional outcome and response variability origins.
      ]). This heterogeneity in the applied procedures has to be considered when comparing NIBS response rates across studies.

      2.1 Clinical and behavioral characteristics

      Several studies from different behavioral domains were able to associate standardized clinical scales with the magnitude of the NIBS response. For instance, O'Shea and colleagues were able to establish an association between a higher (better) Fugl-Meyer Assessment score for the upper extremity (FMA-UE) in combination with a longer time interval since the stroke; and a larger responsiveness towards a cathodal (inhibitory) contralesional M1 tDCS protocol in their cohort (N = 13) of chronic stroke patients [
      • O'Shea J.
      • Boudrias M.-H.
      • Stagg C.J.
      • Bachtiar V.
      • Kischka U.
      • Ju Blicher
      • et al.
      Predicting behavioural response to TDCS in chronic motor stroke.
      ]. The detected two-factor association is in line with emerging evidence from imaging studies suggesting a remaining supportive role of the contralesional M1 for severely impaired patients well into the chronic stage [
      • Grefkes C.
      • Ward N.S.
      Cortical reorganization after stroke: how much and how functional?.
      ], which likely was disturbed by the cathodal tDCS application.
      Another example is taken from the language domain. Norise and colleagues studied patients with first time, single, left-hemispheric, chronic stroke (N = 9) applying ten sessions of an individualized tDCS protocol [
      • Norise C.
      • Sacchetti D.
      • Hamilton R.
      Transcranial direct current stimulation in post-stroke chronic aphasia: the impact of baseline severity and task specificity in a pilot sample.
      ]. The applied protocol was chosen from four possible active conditions (anodal tDCS or cathodal tDCS applied to left or right frontal lobe) based on which elicited the best response in a montage-finding phase. Responsiveness to the best individualized, active stimulation condition was associated with a fluency item of the Boston Diagnostic Aphasia Examination at baseline, indicating larger responsiveness for patients with a stronger baseline impairment. The authors speculate that the larger responsiveness of severely impaired patients might be due to an extended “recovery window” well into the chronic phase and resulting higher susceptibility for the NIBS intervention.
      A further important feature determining responsiveness might be the location of the lesion. This association was for instance established in the important study of Ameli and colleagues [
      • Ameli M.
      • Grefkes C.
      • Kemper F.
      • Riegg F.P.
      • Rehme A.K.
      • Karbe H.
      • et al.
      Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
      ]. The authors applied a high-frequency rTMS protocol to ipsilesional M1 studying effects on finger and hand tapping movements in stroke patients (N = 29) from different recovery stages (acute to chronic). For this purpose, the patients were classified, using high-resolution structural magnetic resonance imaging (MRI), into a subcortical lesion group and a group with additional cortical lesions. The analysis indicated a significant interaction between lesion location and intervention, suggesting a higher susceptibility for patients with a subcortical-only lesion pattern. This suggests that the functional integrity of the stimulation site might be critical. The notion is supported by computational modeling work documenting that the altered electrical properties in stroke regions affect the stimulation currents in magnitude, location, and orientation; and these alterations may result in variable response rates across patients [
      • O'Brien A.T.
      • Amorim R.
      • Rushmore R.J.
      • Eden U.
      • Afifi L.
      • Dipietro L.
      • et al.
      Motor cortex neurostimulation technologies for chronic post-stroke pain: implications of tissue damage on stimulation currents.
      ].

      2.2 TMS-based parameters

      Different TMS-based parameters have been associated with NIBS susceptibility. A simple and straightforward approach is to assess the presence versus absence of motor evoked potentials (MEPs) in the affected limb. The approach is integrated in workflows aiming at predicting recovery of motor function after stroke, such as the PREP2 algorithm [
      • Stinear C.M.
      • Byblow W.D.
      • Ackerley S.J.
      • Smith M.
      • Borges V.M.
      • Barber P.A.
      PREP2: a biomarker-based algorithm for predicting upper limb function after stroke.
      ]. In addition, presence of MEP response has been related with NIBS response rates. For example, Lee and colleagues were able to associate a presence of MEP with good responsiveness to a high-frequency rTMS protocol applied for two weeks to the affected hemisphere of subacute stroke patients (N = 29) [
      • Lee J.H.
      • Kim S.B.
      • Lee K.W.
      • Kim M.A.
      • Lee S.J.
      • Choi S.J.
      Factors associated with upper extremity motor recovery after repetitive transcranial magnetic stimulation in stroke patients.
      ]. Furthermore, TMS can be used to study surrogate parameters of neurotransmission [
      • Chen R.
      Interactions between inhibitory and excitatory circuits in the human motor cortex.
      ]. In this regard, assessments of GABAA-ergic neurotransmission, for instance via studying the paired-pulse short intracortical inhibition protocol, likely contains valuable information as GABA serves a critical role in mediating motor learning processes, stroke recovery, and tDCS effects [
      • Bachtiar V.
      • Stagg C.J.
      The role of inhibition in human motor cortical plasticity.
      ,
      • Liuzzi G.
      • Hörniß V.
      • Lechner P.
      • Hoppe J.
      • Heise K.
      • Zimerman M.
      • et al.
      Development of movement-related intracortical inhibition in acute to chronic subcortical stroke.
      ,
      • Hummel F.C.
      • Steven B.
      • Hoppe J.
      • Heise K.
      • Thomalla G.
      • Cohen L.G.
      • et al.
      Deficient intracortical inhibition (SICI) during movement preparation after chronic stroke.
      ]. An example of this approach was the study of Zimerman and colleagues, who showed a relationship between contralesional M1 cathodal tDCS-induced short intracortical inhibition modulation and tDCS-responsiveness quantified via the online improvement in a hand skill learning task in chronic stroke patients (N = 12) [
      • Zimerman M.
      • Heise K.F.
      • Hoppe J.
      • Cohen L.G.
      • Gerloff C.
      • Hummel F.C.
      Modulation of training by single-session transcranial direct current stimulation to the intact motor cortex enhances motor skill acquisition of the paretic hand.
      ].

      2.3 MRI-based parameters

      Stroke has the characteristics of a network disorder [
      • Koch P.J.
      • Hummel F.C.
      Toward precision medicine: tailoring interventional strategies based on noninvasive brain stimulation for motor recovery after stroke.
      ,
      • Grefkes C.
      • Fink G.R.
      Reorganization of cerebral networks after stroke: new insights from neuroimaging with connectivity approaches.
      ,
      • Koch P.
      • Schulz R.
      • Hummel F.C.
      Structural connectivity analyses in motor recovery research after stroke.
      ]. Structural and functional alterations to the network can be well captured with MRI-based imaging techniques. A key structure for successful recovery of motor function is the cortico-spinal tract [
      • Lin David J.
      • Cloutier Alison M.
      • Erler Kimberly S.
      • Cassidy Jessica M.
      • Snider Samuel B.
      Ranford Jessica, et al. Corticospinal Tract Injury Estimated From Acute Stroke Imaging Predicts Upper Extremity Motor Recovery After Stroke.
      ], whose role for determining tDCS susceptibility has been studied by Lindenberg and colleagues using diffusion tensor imaging techniques [
      • Lindenberg R.
      • Zhu L.L.
      • Rüber T.
      • Schlaug G.
      Predicting functional motor potential in chronic stroke patients using diffusion tensor imaging.
      ]. In their seminal study of chronic stroke patients (N = 15), they were able to describe an association between better structural integrity measures of the cortico-spinal tract and larger gains in motor function following a combined protocol of bihemispheric (M1-M1) tDCS and physical/occupational therapy. An alternative approach is the evaluation of effective connectivity measures for example applying the dynamic causal modeling technique. Utilizing this approach, Diekhoff-Krebs and colleagues found that in chronic stroke patients (N = 14), both stronger positive coupling between ipsilesional supplementary motor area and stronger negative coupling between ipsilesional M1 and contralesional M1 were associated with better motor response to intermittent theta-burst stimulation of ipsilesional M1 [
      • Diekhoff-Krebs S.
      • Pool E.-M.
      • Sarfeld A.-S.
      • Rehme A.K.
      • Eickhoff S.B.
      • Fink G.R.
      • et al.
      Interindividual differences in motor network connectivity and behavioral response to iTBS in stroke patients.
      ]. Promising imaging-derived parameters for future associational studies with NIBS responsiveness include metrics characterizing cross-network interactions [
      • Schulz R.
      • Park E.
      • Lee J.
      • Chang W.H.
      • Lee A.
      • Kim Y.-H.
      • et al.
      Interactions between the corticospinal tract and premotor-motor pathways for residual motor output after stroke.
      ] or parameters derived from whole-brain analysis, for example studying “connectome fingerprinting” [
      • Finn E.S.
      • Shen X.
      • Scheinost D.
      • Rosenberg M.D.
      • Huang J.
      • Chun M.M.
      • et al.
      Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity.
      ,
      • Ovadia-Caro S.
      • Khalil A.A.
      • Sehm B.
      • Villringer A.
      • Nikulin V.V.
      • Nazarova M.
      Predicting the response to non-invasive brain stimulation in stroke.
      ,
      • Egger P
      • Evangelista GG
      • Koch PJ
      • Park C
      • Levin-Gleba L
      • Girard G
      • et al.
      Disconnectomics of the rich club impacts motor recovery after stroke.
      ,
      • Koch PJ
      • Park C
      • Girard G
      • Beanato E
      • Egger P
      • Evangelista GG
      • et al.
      The structural connectome as a biomarker for motor recovery after stroke: predicting natural recovery in the acute stage.
      ].

      2.4 Combined approaches

      Finally, different assessment domains may be combined in one statistical model. A recent example originates from the study by Kuo and colleagues of subacute stroke patients (N = 18) [
      • Kuo I.-J.
      • Tang C.-W.
      • Tsai Y.-A.
      • Tang S.-C.
      • Lin C.-J.
      • Hsu S.-P.
      • et al.
      Neurophysiological signatures of hand motor response to dual-transcranial direct current stimulation in subacute stroke: a TMS and MEG study.
      ]. The authors assessed a set of clinical, TMS-based and magnetoencephalography-based metrics. Using a stepwise multiple regression approach with backward elimination, the authors identified the baseline ipsilateral silent period ratio as more influential than five other independent variables (age, gender, baseline FMA-UE score, baseline ipsilesional-to-contralesional MEP ratio, baseline ipsilesional-to-contralesional alpha event-related synchronization ratio) in determining responsiveness towards a combined intervention of M1-M1 tDCS and paretic hand exercise. Specifically, larger ipsilateral silent period ratios were associated with a lower responsiveness to tDCS. An advantage of such approaches is that they potentially account for several mechanistically independent factors. However, a clear downside is that the winning model depends heavily on the feature selection procedure.

      2.5 Perturbation probe-based approaches

      A complementary approach to the described associational studies is the use of TMS-induced “virtual lesion” (VL) experiments, which allows for studying causal brain-behavior relationships [
      • Pascual-Leone A.
      • Bartres-Faz D.
      • Keenan J.P.
      Transcranial magnetic stimulation: studying the brain-behaviour relationship by induction of “virtual lesions.
      ]. The VL approach takes advantage of the capability of single TMS pulses or short trains of repetitive pulses to temporarily disrupt the functionality of a given cortical target. By doing so, it is possible to quantify the impact on behavior or imaging metrics of “shutting off” the target. The approach has provided valuable insights for better understanding of brain network alterations post-stroke. For instance, Lotze and colleagues showed in a VL experiment that contralesional motor areas played a supportive role in organizing complex finger movements in the studied sample (N = 7) of narrowly selected (only internal capsule lesions), well recovered chronic stroke patients. This demonstrates that contralesional motor areas do not always have a maladaptive role as suggested by interhemispheric competition models [
      • Hummel F.C.
      • Cohen L.G.
      Non-invasive brain stimulation: a new strategy to improve neurorehabilitation after stroke?.
      ,
      • Lotze M.
      • Markert J.
      • Sauseng P.
      • Hoppe J.
      • Plewnia C.
      • Gerloff C.
      The role of multiple contralesional motor areas for complex hand movements after internal capsular lesion.
      ]. In a more recent example, Hensel and colleagues showed that “virtually lesioning” the contralesional anterior intraparietal sulcus improved performance on a tapping task, suggesting a maladaptive role of this region in the studied sample of acute first-time stroke patients (N = 14) [
      • Hensel L.
      • Tscherpel C.
      • Freytag J.
      • Ritter S.
      • Rehme A.K.
      • Volz L.J.
      • et al.
      Connectivity-related roles of contralesional brain regions for motor performance early after stroke.
      ]. Furthermore, the VL approach has shown promise in tracking the longitudinal change of the recovery-facilitating role of particular brain regions. A study by Tscherpel and colleagues [
      • Tscherpel C.
      • Hensel L.
      • Lemberg K.
      • Vollmer M.
      • Volz L.J.
      • Fink G.R.
      • et al.
      The differential roles of contralesional frontoparietal areas in cortical reorganization after stroke.
      ] of patients with first-ever ischemic stroke in the left hemisphere and mild to moderate motor deficit (N = 14) found that the time-sensitivity of interference to contralesional frontoparietal areas is region-specific. Another study by the same group [
      • Tscherpel C.
      • Dern S.
      • Hensel L.
      • Ziemann U.
      • Fink G.R.
      • Grefkes C.
      Brain responsivity provides an individual readout for motor recovery after stroke.
      ] found that slow and simple electroencephalogram responses to TMS were associated with both severe motor impairment and poor motor recovery of stroke patients (N = 25).
      The described VL approach could also allow for assigning individual patients to a tailored NIBS intervention when applied as a baseline probe preceding the actual interventional phase. As described in a detailed review by Morishita and colleagues, a VL probe could be used to determine the functional role of the contralesional M1 for an individual patient in a recovery-phase-specific manner [
      • Morishita T.
      • Hummel F.C.
      Non-invasive brain stimulation (NIBS) in motor recovery after stroke: concepts to increase efficacy.
      ]. In the described study concept, patients would receive excitatory NIBS to the contralesional M1 following a detrimental response to the VL probe. Conversely, a beneficial response would result in the application of an inhibitory NIBS protocol to the contralesional M1. As illustrated, the VL approach could be one strategy to circumvent the dilemma of translating findings from associational studies to individual out-of-sample patients when applied before the allocation of an individual patient to a tailored intervention.

      3. Part 2: predictive modeling

      As we saw in Part 1, many studies have examined responsiveness of stroke patients to NIBS interventions. Since research on NIBS for treatment of stroke is still in a preclinical stage, these studies are primarily focused on finding associations and correlations between responsiveness to NIBS, broadly defined, and various individual factors.

      3.1 Associational studies

      Data mining studies, which seek to find associations and correlations between variables of interest are a necessary prerequisite for precision medicine, because they generate the insights and hypotheses that may eventually lead to more robust claims. The only rule of analytics is that any insights derived from a sample can only be considered valid on that sample, and attempts to generalize the insights to a larger population must be made with an open mind. This is particularly true when samples are small and/or biased, as is the case in most studies cited in Part 1.

      3.2 Precision medicine

      The ultimate goal of much healthcare research today is the establishment of precision medicine, defined by the United States National Institutes of Health and Precision Medicine Initiative as “an emerging approach for disease treatment and prevention that takes into account individual variability in genes, environment and lifestyle for each person.” Precision medicine requires mathematical models of disease to provide actionable information to patients and caregivers (see Fig. 1A), which in turn requires the ability of such models to predict with reasonable accuracy the course of recovery of a patient (or group) based on demographic or clinical characteristics of the patient (or group). In short, precision medicine requires the ability to make predictive models. Importantly, such models should be robust, meaning that they should perform well on all subsets of the target population. Mathematical models in healthcare can largely be divided into rules-based and machine learning models.
      Fig. 1
      Fig. 1Rules-based and machine learning models in healthcare. The problem to solve is the following (A): given a patient with certain characteristics, the decision-making machine should output a recommended course of treatment or a prediction of responsiveness to a particular course of treatment. The decision-making machine may be produced in a rules-based manner (B) in which human experts explicitly hard-code the rules. Alternatively, it may be produced in an ML-based manner (C) in which humans provide data and an algorithm infers the rules. Please note the greater number of knobs in (C) compared to (B); the number of knobs, which represent model parameters, is a reasonable proxy for model complexity. Supervised ML models (D, left column) are trained on data affixed with labels; if the labels are continuous numbers, we refer to a regression problem, while if they are discrete categories, we refer to a classification problem. Unsupervised ML models (D, right column) are trained on data without labels; common problems include finding clusters in the data, and dimensionality reduction, i.e. reducing the number of features in the data while retaining most of the information. A key metric for ML models is whether they perform equally well on data used for training as on other data; failure to do so is known as overfitting (E). To illustrate common causes of overfitting, we plotted some deliberately uncorrelated random x and y data (E, left column) and fit polynomial equations to the data (E, right column). In each subplot, the in-sample data was denoted by black dots and the out of sample data by blue crosses, while the polynomials increase in complexity from linear to quadratic to cubic. Overfitting, therefore, is the extent to which the fitted models, denoted by red lines, model the black dots far better than the blue crosses. Please note that overfitting is least prominent with large sample sizes and simpler models. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

      3.3 Rules-based models

      A rules-based model (see Fig. 1B) is one in which decision rules are explicitly hard-coded by experts relying on their knowledge and experience. Examples of rules-based models include the various triage protocols adopted by hospital emergency departments, many of which are described in a review by Bazyar and colleagues [
      • Bazyar J.
      • Farrokhi M.
      • Khankeh H.
      Triage systems in mass casualty incidents and disasters: a review study with A worldwide approach.
      ]. These models are typically expressed as logical flow charts.
      Such models are typically transparent, and can thus be easily passed through the filter of common sense. They are easy to use, providing an unequivocal answer with great speed and minimal requirements for data infrastructure.
      On the downside, their data-agnostic and non-evidence-based nature makes them potentially biased by the opinions of the experts who created them. For instance, as Jenkins and colleagues point out, it is possible that triage protocols initially developed by the military may incorporate decisions geared towards advancing mission objectives in addition to purely medical considerations [
      • Jenkins J.L.
      • McCarthy M.L.
      • Sauer L.M.
      • Green G.B.
      • Stuart S.
      • Thomas T.L.
      • et al.
      Mass-casualty triage: time for an evidence-based approach.
      ].

      3.4 Machine learning models

      A machine learning (ML) model is one in which decision rules are implicitly inferred by computer algorithms from data provided to the computer (see Fig. 1C) [
      • Mitchell T.
      Machine learning.
      ]. In the same way that human learning is the process of gradually improving performance at a task by sensory input and practice, machine learning is the process of computers improving performance through data and algorithms. ML models are characterized by an overall architecture and hyperparameters chosen by humans; and parameters, which are automatically fit to, or “learned” from human-provided data.
      Most ML models can be categorized as either supervised or unsupervised. Supervised ML seeks to create models of the relationship between predictor variables X and response variables Y. The term “supervised” refers to the fact that the responses can serve as a ground truth to which predictions can be compared; the Y is “supervising” the X. Common problems for supervised ML are regression (response variables are continuous numbers) and classification (response variables are discrete categories). Unsupervised ML seeks to find patterns within the data set, i.e. we have only the X, not the Y. Common problems for unsupervised ML are clustering (grouping data points according to a notion of similarity) and dimensionality reduction (eliminating or combining features in the data while preserving most of the information within). Illustrations of these types of ML problems are given in Fig. 1D.
      On the downside, the well-known computer science saying “garbage in, garbage out” distills a fundamental truth about all ML models: one cannot outsource thoughtfulness and common sense to algorithms. The quality of the final model depends heavily on the choice of model architecture and hyperparameters that make up the initial model, and on the quality of the data used to train the model. In particular, attempting to create a quality ML model without datasets that are sufficiently large and unbiased is a fool's errand.

      3.5 ML in stroke treatment

      ML has great potential in assisting stroke treatment. Tozlu and colleagues found that ML algorithms such as elastic net or random forest applied to demographic, clinical and imaging variables show some promise in predicting stroke patients’ (N = 102) motor function after a rTMS intervention, which is a regression problem [
      • Tozlu C.
      • Edwards D.
      • Boes A.
      • Labar D.
      • Tsagaris K.Z.
      • Silverstein J.
      • et al.
      Machine Learning methods predict individual upper limb motor impairment following therapy in chronic stroke.
      ]. Furthermore, it has been found that support vector machines showed promise in the classification problem of predicting whether stroke patients would have good or poor motor outcome, using functional MRI data (Rehme and colleagues, N = 21 [
      • Rehme A.K.
      • Volz L.J.
      • Feis D.-L.
      • Eickhoff S.B.
      • Fink G.R.
      • Grefkes C.
      Individual prediction of chronic motor outcome in the acute post-stroke stage: behavioral parameters versus functional imaging.
      ]) or structural MRI data (Koch and colleagues, N = 92 [
      • Koch PJ
      • Park C
      • Girard G
      • Beanato E
      • Egger P
      • Evangelista GG
      • et al.
      The structural connectome as a biomarker for motor recovery after stroke: predicting natural recovery in the acute stage.
      ]).

      3.6 Out-of-sample generalizability

      As mentioned earlier, when doing predictive modeling as opposed to data mining, robustness of a model, i.e. its ability to perform well on out-of-sample data as well as on in-sample data, is the true test of the validity of a model (see Fig. 1E). Indeed, a model which performed well on sample data, but poorly out-of-sample, would be misleading and therefore probably worse than no model at all in the view of the general population. Three phenomena can cause this to occur. The first is overfitting, in which a model fits the noise in the data more than the underlying population, as illustrated in the right half of Fig. 1E. The second is sampling bias, the phenomenon whereby a model trained on biased data will produce a similarly biased outcome. For instance, in the left half of Fig. 1E, the x and y sample data are uncorrelated, which can occur even if they are correlated in the broader population. The third is data snooping, in which the ability of a subset of the data to assess model quality is compromised if that data has affected any step of the learning process.

      3.7 Avoiding overfitting by regularization and choosing simple models

      A model's complexity is largely a function of how much flexibility it has to contort itself to fit to the data. Thus, researchers should adopt the principle of Occam's razor, which in the context of ML states that simpler models are to be preferred. This is well illustrated in Fig. 1E, which shows the more complex models overfitting the training data at the expense of wildly missing the non-training data.
      Models are trained by fitting the parameters to minimize some quantity considered a proxy for “model badness.” If Occam's razor suggests a syllogism between “complex” and “bad,” then it makes sense to incorporate complexity into our definition of “badness,” thereby causing the training process to favor simpler versions of a given model. This penalization of complexity is known as regularization. Some well-known examples regularization in linear regression include lasso regression, which drives many coefficients to equal zero, ridge regression, which drives many coefficients to be close to zero, or elastic net, which is a hybrid of lasso and ridge.
      Regularization, as well as a preference for simpler model architectures to begin with, are powerful tools against overfitting, which researchers should keep in mind.

      3.8 Combating sampling bias by data sharing

      Combating sampling bias is more straightforward, albeit likely a harder pill for the research community to swallow: it requires that data, gathered at considerable expense, be freely shared across labs. While this may seem counter to the competitive aspect of modern research, encouraging trends in this direction are clearly visible: the number of PubMed articles matching the term “open data” is growing steadily, from 6122 in 2010 to 19,300 in 2020. Other forces moving the community towards more open data are outlined by McKiernan and colleagues [
      • McKiernan E.C.
      • Bourne P.E.
      • Brown C.T.
      • Buck S.
      • Kenall A.
      • Lin J.
      • et al.
      How open science helps researchers succeed.
      ].

      3.9 Avoiding data snooping by train-validate-test separation

      While it is common to see the terms “validation” and “testing” used interchangeably, as both refer to data not used to fit the model, non-training data is in fact required for two mutually exclusive purposes. As Kuhn and Johnson warn, while the parameters are fit using training data, tuning of hyperparameters, which is done before fitting the model using non-training data (to avoid data snooping) is also a part of the learning process [
      • Kuhn M.
      • Johnson K.
      Applied predictive modeling.
      ]. Thus, the learning process is really two nested loops: the outer loop explores the space of possible hyperparameters, and for each set of hyperparameters, the inner loop fits the model parameters on the training data and evaluates model quality on some non-training data. To avoid data snooping, this non-training data cannot be used to evaluate the quality of the final model, once the hyperparameters have been chosen and the parameters fit. Thus, the aforementioned non-training data is typically called validation data, and another subset, typically called the test set, of non-training data must be kept in reserve, not to be touched until we need to evaluate the quality of the final model fit.
      Avoiding data snooping therefore requires us to either forgo hyperparameter tuning altogether, which is a myopic way to do ML, or keep strict separation of training, test and validation datasets. For more on hyperparameters, we refer the reader to the Supplementary Online Material.

      3.10 Limitations

      The challenge of data aggregation is compounded by the fact that combining data from multiple studies may introduce nuisance covariates that may (at least partially) offset the benefits of more representative samples. As an example, consider two studies of rTMS in stroke patients, one of which is conducted at subacute stage in country A, and the other of which is conducted in chronic stage in country B. Then longitudinal changes in responsiveness may reflect genuine recovery or simply differences in the healthcare systems of countries A and B. Other nuisance covariates may include whether one is considering immediate or long-term effects of NIBS, or which traditional therapies the NIBS served as an adjunct to. Such covariates must be taken seriously in any attempt to aggregate multiple data sets.

      3.11 Summary

      In summary, ML has great potential to leverage large amounts of data to support medical practitioners by making predictions of patients’ responses to treatments. However, it is important not to get swept up in the hype surrounding ML, and to realize that if used without sufficient care, it is likely to lead researchers into serious error by proposing models whose apparent predictive power is little more than a mirage. As proven in landmark work of Ioannidis [
      • Ioannidis J.P.A.
      Why most published research findings are false.
      ], a significant part of all published research is false or not reproduced, and we believe that as ML makes its way into ever more fields of inquiry, the proportion of published results that is not reproducible or false is likely to increase unless great care is taken. It is our hope that neuroscience researchers intending to use ML for predicting will always keep in mind both its pitfalls, and some ways to avoid them.
      To avoid overfitting, simple models should be chosen to begin with, and further simplify the ones they chose via regularization. To avoid data snooping, one should be meticulous about differentiating validation and testing datasets. Finally, to avoid sample bias, one should make efforts to gather sufficiently large and unbiased datasets; if such is not possible, one should pool the data with data gathered by others; and if such is not possible, one should avoid ML altogether.

      4. Conclusions

      The development of predictive models for responsiveness of stroke survivors to NIBS (or any other interventional strategy) has the potential to guide personalized application protocols in the future. This could be key to reduce the heterogeneity of outcomes and maximize the individual treatment response associated with conventional NIBS protocols. However, further testing is needed to demonstrate the benefits of this novel approach.
      It is important to be cautious not to make premature and unsubstantiated claims on prediction. As discussed above, it is common for detected in-sample associations to fail to generalize due to model overfitting. The most important contributor to overfitting is sampling bias, which is unavoidable when one is restricted to limited numbers of patients in one study site. Unfortunately, due to the high logistical and financial costs associated with most clinical interventional neurotechnology-associated trials, it is challenging to aggregate a sufficiently large data set at a single research unit. Data sharing across units could overcome this hurdle [
      • McKiernan E.C.
      • Bourne P.E.
      • Brown C.T.
      • Buck S.
      • Kenall A.
      • Lin J.
      • et al.
      How open science helps researchers succeed.
      ]. Contrary to common worries that data sharing entails risks to proper attribution of credit and funding, researchers who practice open science benefit from clear documentation, enhanced preservation, data curation, reproducibility, transparency and more citations of their research [
      • McKiernan E.C.
      • Bourne P.E.
      • Brown C.T.
      • Buck S.
      • Kenall A.
      • Lin J.
      • et al.
      How open science helps researchers succeed.
      ]. Based on these considerations, we would like to echo calls for open science and data sharing emerging from major funding agencies [
      • McKiernan E.C.
      • Bourne P.E.
      • Brown C.T.
      • Buck S.
      • Kenall A.
      • Lin J.
      • et al.
      How open science helps researchers succeed.
      ] and the NIBS community [
      • Koch PJ
      • Park C
      • Girard G
      • Beanato E
      • Egger P
      • Evangelista GG
      • et al.
      The structural connectome as a biomarker for motor recovery after stroke: predicting natural recovery in the acute stage.
      ,
      • Grossman P.
      • Alekseichuk I.
      • de Lara G.
      • Paneri K.
      • Kunz P.
      • Turi Z.
      • et al.
      Transcranial direct current stimulation studies open database (tDCS-OD).
      ,
      • Belardinelli P.
      • Biabani M.
      • Blumberger D.M.
      • Bortoletto M.
      • Casarotto S.
      • David O.
      • et al.
      Reproducibility in TMS-EEG studies: a call for data sharing, standard procedures and effective experimental control.
      ]. If successful, we believe open science initiatives would provide a solid framework for accomplishing the task of creating robust predictive models for NIBS responsiveness.

      Declarations of interest

      None.

      CRediT author statement

      Maximilian J. Wessel: Conceptualization, Investigation, Writing - Original Draft, Writing - Review & Editing, Project administration, Funding acquisition. Philip Egger: Conceptualization, Investigation, Writing - Original Draft, Writing - Review & Editing, Visualization. Friedhelm C. Hummel: Conceptualization, Writing - Review & Editing, Supervision, Funding acquisition.

      Acknowledgements

      The research was partially funded by the Strategic Focus Area “Personalized Health and Related Technologies ( PHRT, #2017–205 )” of the ETH Domain (CH) to F.C.H., the Novartis Research Foundation—FreeNovation , (Basel, CH) to M.J.W., the Wyss Center for Bio and Neuroengineering (ecoss WCP024; Genève, Switzerland) to F.C.H., the Defitech Foundation (Morges, CH) to F.C.H. and the Bertarelli Foundation—Catalyst program (Gstaad, CH) to F.C.H.

      Appendix A. Supplementary data

      Panel 1: non-invasive brain stimulation (NIBS) techniques

      Transcranial direct current stimulation (tDCS)
      Repetitive transcranial magnetic stimulation (rTMS)
      • Mechanism: modulation underlying neuronal tissue through focal, brief, time-varying currents generated via electromagnetic induction
      • Modes
      • Conventional: excitatory – high-frequency ≥ 5 Hz (HF), inhibitory – low-frequency 0.2–1 Hz (LF).
      • Patterned: excitatory – intermittent theta-burst stimulation (iTBS), inhibitory – continuous theta-burst stimulation (cTBS).
      • Stimulation parameters: intensity [% of MSO], pulse-shape, current-direction, coil-type, frequency [Hz], number of pulses, train-duration [s], inter-train-interval [s], burst-configuration
      • Further reading [
        • Hallett M.
        Transcranial magnetic stimulation: a primer.
        ,
        • Rossi S.
        • Antal A.
        • Bestmann S.
        • Bikson M.
        • Brewer C.
        • Brockmöller J.
        • et al.
        Safety and recommendations for TMS use in healthy subjects and patient populations, with updates on training, ethical and regulatory issues: expert Guidelines.
        ]
      Emerging technologies
      Transcranial focused ultrasound stimulation (tFUS)

      Panel 2: predictive modeling concepts and terminology

      Machine learning (ML): the process of teaching computers to perform tasks by creating useful models of the world.
      Parameter: Component of an ML model which is learned from data.
      Hyperparameter: Component of an ML model which is not learned from data.
      Supervised ML: The task of finding a relationship between one or more predictor variables X and one or more response variables Y.
      Unsupervised ML: The task of finding patterns within a dataset X, without making predictions that can be checked against (or “supervised by”) response variables.
      Data mining: The process of combing through datasets to extract insights. Synonymous with analytics, explanatory data analysis or associational study. Strong claims should be avoided and an open mind kept as insights may not generalize out of sample.
      Predictive modeling: The process of creating models for the purpose of making individualized predictions. Claims of predictive power should be avoided when sample sizes are small and/or derived from a single site.
      Overfitting: The phenomenon whereby a model learns more from noise in the data than from the population distribution. A particular threat when 1) datasets are small and/or biased, 2) datasets contain many predictor variables, or 3) excessively complex ML models are used.
      Cross-validation: The process of partitionin2yg the dataset into several slices, or folds, and repeatedly holding one-fold out from training, in order to test on the held-out fold.
      Leave-one-out cross-validation: The form of cross-validation in which each fold consists of a single data point. Not recommended unless dealing with small datasets.
      Data snooping: The use of training data in any way to test a model, or analogously using the same data to develop and test a hypothesis. A particular threat when creating data preprocessing pipelines, it will make ML models appear better than they are.

      References

        • Feigin V.L.
        • Norrving B.
        • Mensah G.A.
        Global burden of stroke.
        Circ Res. 2017; 120: 439-448https://doi.org/10.1161/CIRCRESAHA.116.308413
        • Dobkin B.H.
        Clinical practice. Rehabilitation after stroke.
        N Engl J Med. 2005; 352: 1677-1684https://doi.org/10.1056/NEJMcp043511
        • Kapoor A.
        • Lanctôt K.L.
        • Bayley M.
        • Kiss A.
        • Herrmann N.
        • Murray B.J.
        • et al.
        “Good outcome” isn't good enough: cognitive impairment, depressive symptoms, and social restrictions in physically recovered stroke patients.
        Stroke. 2017; 48: 1688-1690https://doi.org/10.1161/STROKEAHA.117.016728
        • Raffin E.
        • Hummel F.C.
        Restoring motor functions after stroke: multiple approaches and opportunities.
        Neurosci Rev J Bringing Neurobiol Neurol Psychiatry. 2018; 24: 400-416https://doi.org/10.1177/1073858417737486
        • Hummel F.C.
        • Cohen L.G.
        Non-invasive brain stimulation: a new strategy to improve neurorehabilitation after stroke?.
        Lancet Neurol. 2006; 5: 708-712https://doi.org/10.1016/S1474-4422(06)70525-7
        • Wessel M.J.
        • Zimerman M.
        • Hummel F.C.
        Non-invasive brain stimulation: an interventional tool for enhancing behavioral training after stroke.
        Front Hum Neurosci. 2015; 9: 265https://doi.org/10.3389/fnhum.2015.00265
        • Lefaucheur J.-P.
        • André-Obadia N.
        • Antal A.
        • Ayache S.S.
        • Baeken C.
        • Benninger D.H.
        • et al.
        Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS).
        Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2014; 125: 2150-2206https://doi.org/10.1016/j.clinph.2014.05.021
        • Hummel F.
        • Celnik P.
        • Giraux P.
        • Floel A.
        • Wu W.-H.
        • Gerloff C.
        • et al.
        Effects of non-invasive cortical stimulation on skilled motor function in chronic stroke.
        Brain J Neurol. 2005; 128: 490-499https://doi.org/10.1093/brain/awh369
        • Hummel F.C.
        • Voller B.
        • Celnik P.
        • Floel A.
        • Giraux P.
        • Gerloff C.
        • et al.
        Effects of brain polarization on reaction times and pinch force in chronic stroke.
        BMC Neurosci. 2006; 7: 73https://doi.org/10.1186/1471-2202-7-73
        • Khedr E.M.
        • Ahmed M.A.
        • Fathy N.
        • Rothwell J.C.
        Therapeutic trial of repetitive transcranial magnetic stimulation after acute ischemic stroke.
        Neurology. 2005; 65: 466-468https://doi.org/10.1212/01.wnl.0000173067.84247.36
        • Mansur C.G.
        • Fregni F.
        • Boggio P.S.
        • Riberto M.
        • Gallucci-Neto J.
        • Santos C.M.
        • et al.
        A sham stimulation-controlled trial of rTMS of the unaffected hemisphere in stroke patients.
        Neurology. 2005; 64: 1802-1804https://doi.org/10.1212/01.WNL.0000161839.38079.92
        • Zimerman M.
        • Heise K.F.
        • Hoppe J.
        • Cohen L.G.
        • Gerloff C.
        • Hummel F.C.
        Modulation of training by single-session transcranial direct current stimulation to the intact motor cortex enhances motor skill acquisition of the paretic hand.
        Stroke. 2012; 43: 2185-2191https://doi.org/10.1161/STROKEAHA.111.645382
        • Hesse S.
        • Waldner A.
        • Mehrholz J.
        • Tomelleri C.
        • Pohl M.
        • Werner C.
        Combined transcranial direct current stimulation and robot-assisted arm training in subacute stroke patients: an exploratory, randomized multicenter trial.
        Neurorehabilitation Neural Repair. 2011; 25: 838-846https://doi.org/10.1177/1545968311413906
        • Lefaucheur J.-P.
        • Aleman A.
        • Baeken C.
        • Benninger D.H.
        • Brunelin J.
        • Di Lazzaro V.
        • et al.
        Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): an update (2014-2018).
        Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2020; 131: 474-528https://doi.org/10.1016/j.clinph.2019.11.002
        • Lefaucheur J.-P.
        • Antal A.
        • Ayache S.S.
        • Benninger D.H.
        • Brunelin J.
        • Cogiamanian F.
        • et al.
        Evidence-based guidelines on the therapeutic use of transcranial direct current stimulation (tDCS).
        Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2017; 128: 56-92https://doi.org/10.1016/j.clinph.2016.10.087
        • Elsner B.
        • Kugler J.
        • Pohl M.
        • Mehrholz J.
        Transcranial direct current stimulation (tDCS) for improving activities of daily living, and physical and cognitive functioning, in people after stroke.
        Cochrane Database Syst Rev. 2020; 11: CD009645https://doi.org/10.1002/14651858.CD009645.pub4
        • Cunningham D.A.
        • Potter-Baker K.A.
        • Knutson J.S.
        • Sankarasubramanian V.
        • Machado A.G.
        • Plow E.B.
        Tailoring brain stimulation to the nature of rehabilitative therapies in stroke: a conceptual framework based on their unique mechanisms of recovery.
        Phys Med Rehabil Clin. 2015; 26: 759-774https://doi.org/10.1016/j.pmr.2015.07.001
        • Koch P.J.
        • Hummel F.C.
        Toward precision medicine: tailoring interventional strategies based on noninvasive brain stimulation for motor recovery after stroke.
        Curr Opin Neurol. 2017; 30: 388-397https://doi.org/10.1097/WCO.0000000000000462
        • Leuchter A.F.
        • Corlier J.
        A precision medicine approach to repetitive Transcranial Magnetic Stimulation (rTMS).
        Brain Stimulat. 2018; 11: 463-464https://doi.org/10.1016/j.brs.2018.02.003
        • Wessel M.J.
        • Hummel F.C.
        Non-invasive cerebellar stimulation: a promising approach for stroke recovery?.
        Cerebellum Lond Engl. 2018; 17: 359-371https://doi.org/10.1007/s12311-017-0906-1
        • Koch G.
        • Bonnì S.
        • Casula E.P.
        • Iosa M.
        • Paolucci S.
        • Pellicciari M.C.
        • et al.
        Effect of cerebellar stimulation on gait and balance recovery in patients with hemiparetic stroke: a randomized clinical trial.
        JAMA Neurol. 2019; 76: 170-178https://doi.org/10.1001/jamaneurol.2018.3639
        • Plow E.B.
        • Cunningham D.A.
        • Varnerin N.
        • Machado A.
        Rethinking stimulation of the brain in stroke rehabilitation: why higher motor areas might be better alternatives for patients with greater impairments.
        Neurosci Rev J Bringing Neurobiol Neurol Psychiatry. 2015; 21: 225-240https://doi.org/10.1177/1073858414537381
        • Cunningham D.A.
        • Varnerin N.
        • Machado A.
        • Bonnett C.
        • Janini D.
        • Roelle S.
        • et al.
        Stimulation targeting higher motor areas in stroke rehabilitation: a proof-of-concept, randomized, double-blinded placebo-controlled study of effectiveness and underlying mechanisms.
        Restor Neurol Neurosci. 2015; 33: 911-926https://doi.org/10.3233/RNN-150574
        • Zrenner C.
        • Desideri D.
        • Belardinelli P.
        • Ziemann U.
        Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex.
        Brain Stimulat. 2018; 11: 374-389https://doi.org/10.1016/j.brs.2017.11.016
        • Poldrack R.A.
        • Huckins G.
        • Varoquaux G.
        Establishment of best practices for evidence for prediction: a review.
        JAMA Psychiatry. 2020; 77: 534-540https://doi.org/10.1001/jamapsychiatry.2019.3671
        • Nowak D.A.
        • Grefkes C.
        • Dafotakis M.
        • Eickhoff S.
        • Küst J.
        • Karbe H.
        • et al.
        Effects of low-frequency repetitive transcranial magnetic stimulation of the contralesional primary motor cortex on movement kinematics and neural activity in subcortical stroke.
        Arch Neurol. 2008; 65: 741-747https://doi.org/10.1001/archneur.65.6.741
        • Ameli M.
        • Grefkes C.
        • Kemper F.
        • Riegg F.P.
        • Rehme A.K.
        • Karbe H.
        • et al.
        Differential effects of high-frequency repetitive transcranial magnetic stimulation over ipsilesional primary motor cortex in cortical and subcortical middle cerebral artery stroke.
        Ann Neurol. 2009; 66: 298-309https://doi.org/10.1002/ana.21725
        • Hamaguchi T.
        • Yamada N.
        • Hada T.
        • Abo M.
        Prediction of motor recovery in the upper extremity for repetitive transcranial magnetic stimulation and occupational therapy goal setting in patients with chronic stroke: a retrospective analysis of prospectively collected data.
        Front Neurol. 2020; 11: 581186https://doi.org/10.3389/fneur.2020.581186
        • Lindenberg R.
        • Zhu L.L.
        • Rüber T.
        • Schlaug G.
        Predicting functional motor potential in chronic stroke patients using diffusion tensor imaging.
        Hum Brain Mapp. 2012; 33: 1040-1051https://doi.org/10.1002/hbm.21266
        • Lee J.H.
        • Kim S.B.
        • Lee K.W.
        • Kim M.A.
        • Lee S.J.
        • Choi S.J.
        Factors associated with upper extremity motor recovery after repetitive transcranial magnetic stimulation in stroke patients.
        Ann Rehabil Med. 2015; 39: 268-276https://doi.org/10.5535/arm.2015.39.2.268
        • Diekhoff-Krebs S.
        • Pool E.-M.
        • Sarfeld A.-S.
        • Rehme A.K.
        • Eickhoff S.B.
        • Fink G.R.
        • et al.
        Interindividual differences in motor network connectivity and behavioral response to iTBS in stroke patients.
        NeuroImage Clin. 2017; 15: 559-571https://doi.org/10.1016/j.nicl.2017.06.006
        • Nyffeler T.
        • Vanbellingen T.
        • Kaufmann B.C.
        • Pflugshaupt T.
        • Bauer D.
        • Frey J.
        • et al.
        Theta burst stimulation in neglect after stroke: functional outcome and response variability origins.
        Brain J Neurol. 2019; 142: 992-1008https://doi.org/10.1093/brain/awz029
        • O'Shea J.
        • Boudrias M.-H.
        • Stagg C.J.
        • Bachtiar V.
        • Kischka U.
        • Ju Blicher
        • et al.
        Predicting behavioural response to TDCS in chronic motor stroke.
        Neuroimage. 2014; 85: 924-933https://doi.org/10.1016/j.neuroimage.2013.05.096
        • Grefkes C.
        • Ward N.S.
        Cortical reorganization after stroke: how much and how functional?.
        Neurosci Rev J Bringing Neurobiol Neurol Psychiatry. 2014; 20: 56-70https://doi.org/10.1177/1073858413491147
        • Norise C.
        • Sacchetti D.
        • Hamilton R.
        Transcranial direct current stimulation in post-stroke chronic aphasia: the impact of baseline severity and task specificity in a pilot sample.
        Front Hum Neurosci. 2017; 11: 260https://doi.org/10.3389/fnhum.2017.00260
        • O'Brien A.T.
        • Amorim R.
        • Rushmore R.J.
        • Eden U.
        • Afifi L.
        • Dipietro L.
        • et al.
        Motor cortex neurostimulation technologies for chronic post-stroke pain: implications of tissue damage on stimulation currents.
        Front Hum Neurosci. 2016; 10: 545https://doi.org/10.3389/fnhum.2016.00545
        • Stinear C.M.
        • Byblow W.D.
        • Ackerley S.J.
        • Smith M.
        • Borges V.M.
        • Barber P.A.
        PREP2: a biomarker-based algorithm for predicting upper limb function after stroke.
        Ann Clin Transl Neurol. 2017; 4: 811-820https://doi.org/10.1002/acn3.488
        • Chen R.
        Interactions between inhibitory and excitatory circuits in the human motor cortex.
        Exp Brain Res. 2004; 154: 1-10https://doi.org/10.1007/s00221-003-1684-1
        • Bachtiar V.
        • Stagg C.J.
        The role of inhibition in human motor cortical plasticity.
        Neuroscience. 2014; 278: 93-104https://doi.org/10.1016/j.neuroscience.2014.07.059
        • Liuzzi G.
        • Hörniß V.
        • Lechner P.
        • Hoppe J.
        • Heise K.
        • Zimerman M.
        • et al.
        Development of movement-related intracortical inhibition in acute to chronic subcortical stroke.
        Neurology. 2014; 82: 198-205https://doi.org/10.1212/WNL.0000000000000028
        • Hummel F.C.
        • Steven B.
        • Hoppe J.
        • Heise K.
        • Thomalla G.
        • Cohen L.G.
        • et al.
        Deficient intracortical inhibition (SICI) during movement preparation after chronic stroke.
        Neurology. 2009; 72: 1766-1772https://doi.org/10.1212/WNL.0b013e3181a609c5
        • Grefkes C.
        • Fink G.R.
        Reorganization of cerebral networks after stroke: new insights from neuroimaging with connectivity approaches.
        Brain J Neurol. 2011; 134: 1264-1276https://doi.org/10.1093/brain/awr033
        • Koch P.
        • Schulz R.
        • Hummel F.C.
        Structural connectivity analyses in motor recovery research after stroke.
        Ann Clin Transl Neurol. 2016; 3: 233-244https://doi.org/10.1002/acn3.278
        • Lin David J.
        • Cloutier Alison M.
        • Erler Kimberly S.
        • Cassidy Jessica M.
        • Snider Samuel B.
        Ranford Jessica, et al. Corticospinal Tract Injury Estimated From Acute Stroke Imaging Predicts Upper Extremity Motor Recovery After Stroke.
        Stroke. 2019; 50: 3569-3577https://doi.org/10.1161/STROKEAHA.119.025898
        • Schulz R.
        • Park E.
        • Lee J.
        • Chang W.H.
        • Lee A.
        • Kim Y.-H.
        • et al.
        Interactions between the corticospinal tract and premotor-motor pathways for residual motor output after stroke.
        Stroke. 2017; 48: 2805-2811https://doi.org/10.1161/STROKEAHA.117.016834
        • Finn E.S.
        • Shen X.
        • Scheinost D.
        • Rosenberg M.D.
        • Huang J.
        • Chun M.M.
        • et al.
        Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity.
        Nat Neurosci. 2015; 18: 1664-1671https://doi.org/10.1038/nn.4135
        • Ovadia-Caro S.
        • Khalil A.A.
        • Sehm B.
        • Villringer A.
        • Nikulin V.V.
        • Nazarova M.
        Predicting the response to non-invasive brain stimulation in stroke.
        Front Neurol. 2019; 10: 302https://doi.org/10.3389/fneur.2019.00302
        • Egger P
        • Evangelista GG
        • Koch PJ
        • Park C
        • Levin-Gleba L
        • Girard G
        • et al.
        Disconnectomics of the rich club impacts motor recovery after stroke.
        Stroke. 2021; 52: 2115-2124https://doi.org/10.1161/STROKEAHA.120.031541
        • Koch PJ
        • Park C
        • Girard G
        • Beanato E
        • Egger P
        • Evangelista GG
        • et al.
        The structural connectome as a biomarker for motor recovery after stroke: predicting natural recovery in the acute stage.
        Brain. 2021; 144: 2107-2119https://doi.org/10.1093/brain/awab082
        • Kuo I.-J.
        • Tang C.-W.
        • Tsai Y.-A.
        • Tang S.-C.
        • Lin C.-J.
        • Hsu S.-P.
        • et al.
        Neurophysiological signatures of hand motor response to dual-transcranial direct current stimulation in subacute stroke: a TMS and MEG study.
        J Neuroengineering Rehabil. 2020; 17: 72https://doi.org/10.1186/s12984-020-00706-1
        • Pascual-Leone A.
        • Bartres-Faz D.
        • Keenan J.P.
        Transcranial magnetic stimulation: studying the brain-behaviour relationship by induction of “virtual lesions.
        Philos Trans R Soc Lond B Biol Sci. 1999; 354: 1229-1238https://doi.org/10.1098/rstb.1999.0476
        • Lotze M.
        • Markert J.
        • Sauseng P.
        • Hoppe J.
        • Plewnia C.
        • Gerloff C.
        The role of multiple contralesional motor areas for complex hand movements after internal capsular lesion.
        J Neurosci Off J Soc Neurosci. 2006; 26: 6096-6102https://doi.org/10.1523/JNEUROSCI.4564-05.2006
        • Hensel L.
        • Tscherpel C.
        • Freytag J.
        • Ritter S.
        • Rehme A.K.
        • Volz L.J.
        • et al.
        Connectivity-related roles of contralesional brain regions for motor performance early after stroke.
        Cereb Cortex N Y N. 1991 2021; 31: 993-1007https://doi.org/10.1093/cercor/bhaa270
        • Tscherpel C.
        • Hensel L.
        • Lemberg K.
        • Vollmer M.
        • Volz L.J.
        • Fink G.R.
        • et al.
        The differential roles of contralesional frontoparietal areas in cortical reorganization after stroke.
        Brain Stimulat. 2020; 13: 614-624https://doi.org/10.1016/j.brs.2020.01.016
        • Tscherpel C.
        • Dern S.
        • Hensel L.
        • Ziemann U.
        • Fink G.R.
        • Grefkes C.
        Brain responsivity provides an individual readout for motor recovery after stroke.
        Brain. 2020; 143: 1873-1888https://doi.org/10.1093/brain/awaa127
        • Morishita T.
        • Hummel F.C.
        Non-invasive brain stimulation (NIBS) in motor recovery after stroke: concepts to increase efficacy.
        Curr Behav Neurosci Rep. 2017; 4: 280-289https://doi.org/10.1007/s40473-017-0121-x
        • Bazyar J.
        • Farrokhi M.
        • Khankeh H.
        Triage systems in mass casualty incidents and disasters: a review study with A worldwide approach.
        Open Access Maced J Med Sci. 2019; 7: 482-494https://doi.org/10.3889/oamjms.2019.119
        • Jenkins J.L.
        • McCarthy M.L.
        • Sauer L.M.
        • Green G.B.
        • Stuart S.
        • Thomas T.L.
        • et al.
        Mass-casualty triage: time for an evidence-based approach.
        Prehospital Disaster Med. 2008; 23: 3-8https://doi.org/10.1017/S1049023X00005471
        • Mitchell T.
        Machine learning.
        McGraw Hill, 1997
        • Tozlu C.
        • Edwards D.
        • Boes A.
        • Labar D.
        • Tsagaris K.Z.
        • Silverstein J.
        • et al.
        Machine Learning methods predict individual upper limb motor impairment following therapy in chronic stroke.
        Neurorehabilitation Neural Repair. 2020; 34: 428-439https://doi.org/10.1177/1545968320909796
        • Rehme A.K.
        • Volz L.J.
        • Feis D.-L.
        • Eickhoff S.B.
        • Fink G.R.
        • Grefkes C.
        Individual prediction of chronic motor outcome in the acute post-stroke stage: behavioral parameters versus functional imaging.
        Hum Brain Mapp. 2015; 36: 4553-4565https://doi.org/10.1002/hbm.22936
        • McKiernan E.C.
        • Bourne P.E.
        • Brown C.T.
        • Buck S.
        • Kenall A.
        • Lin J.
        • et al.
        How open science helps researchers succeed.
        ELife. 2016; 5e16800https://doi.org/10.7554/eLife.16800
        • Kuhn M.
        • Johnson K.
        Applied predictive modeling.
        Springer Science + Business Media, 2013
        • Ioannidis J.P.A.
        Why most published research findings are false.
        PLoS Med. 2005; 2https://doi.org/10.1371/journal.pmed.0020124
        • Grossman P.
        • Alekseichuk I.
        • de Lara G.
        • Paneri K.
        • Kunz P.
        • Turi Z.
        • et al.
        Transcranial direct current stimulation studies open database (tDCS-OD).
        bioRxiv preprint. 2018; https://doi.org/10.1101/369215
        • Belardinelli P.
        • Biabani M.
        • Blumberger D.M.
        • Bortoletto M.
        • Casarotto S.
        • David O.
        • et al.
        Reproducibility in TMS-EEG studies: a call for data sharing, standard procedures and effective experimental control.
        Brain Stimulat. 2019; 12: 787-790https://doi.org/10.1016/j.brs.2019.01.010
        • Nitsche M.A.
        • Cohen L.G.
        • Wassermann E.M.
        • Priori A.
        • Lang N.
        • Antal A.
        • et al.
        Transcranial direct current stimulation: state of the art 2008.
        Brain Stimulat. 2008; 1: 206-223https://doi.org/10.1016/j.brs.2008.06.004
        • Hallett M.
        Transcranial magnetic stimulation: a primer.
        Neuron. 2007; 55: 187-199https://doi.org/10.1016/j.neuron.2007.06.026
        • Antal A.
        • Alekseichuk I.
        • Bikson M.
        • Brockmöller J.
        • Brunoni A.R.
        • Chen R.
        • et al.
        Low intensity transcranial electric stimulation: safety, ethical, legal regulatory and application guidelines.
        Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2017; 128: 1774-1809https://doi.org/10.1016/j.clinph.2017.06.001
        • Rossi S.
        • Antal A.
        • Bestmann S.
        • Bikson M.
        • Brewer C.
        • Brockmöller J.
        • et al.
        Safety and recommendations for TMS use in healthy subjects and patient populations, with updates on training, ethical and regulatory issues: expert Guidelines.
        Clin Neurophysiol Off J Int Fed Clin Neurophysiol. 2021; 132: 269-306https://doi.org/10.1016/j.clinph.2020.10.003
        • Grossman N.
        • Bono D.
        • Dedic N.
        • Kodandaramaiah S.B.
        • Rudenko A.
        • Suk H.-J.
        • et al.
        Noninvasive deep brain stimulation via temporally interfering electric fields.
        Cell. 2017; 169 (e16): 1029-1041https://doi.org/10.1016/j.cell.2017.05.024
        • Grossman N.
        Modulation without surgical intervention.
        Science. 2018; 361: 461-462https://doi.org/10.1126/science.aau4915
        • Esmaeilpour Z.
        • Kronberg G.
        • Reato D.
        • Parra L.C.
        • Bikson M.
        Temporal interference stimulation targets deep brain regions by modulating neural oscillations.
        Brain Stimulat. 2021; 14: 55-65https://doi.org/10.1016/j.brs.2020.11.007
        • Tyler W.J.
        • Tufail Y.
        • Finsterwald M.
        • Tauchmann M.L.
        • Olson E.J.
        • Majestic C.
        Remote excitation of neuronal circuits using low-intensity, low-frequency ultrasound.
        PloS One. 2008; 3e3511https://doi.org/10.1371/journal.pone.0003511
        • di Biase L.
        • Falato E.
        • Di Lazzaro V.
        Transcranial focused ultrasound (tFUS) and transcranial unfocused ultrasound (tUS) neuromodulation: from theoretical principles to stimulation practices.
        Front Neurol. 2019; 10: 549https://doi.org/10.3389/fneur.2019.00549
        • Rabut C.
        • Yoo S.
        • Hurt R.C.
        • Jin Z.
        • Li H.
        • Guo H.
        • et al.
        Ultrasound technologies for imaging and modulating neural activity.
        Neuron. 2020; 108: 93-110https://doi.org/10.1016/j.neuron.2020.09.003
        • Goto T.
        • Saitoh Y.
        • Hashimoto N.
        • Hirata M.
        • Kishima H.
        • Oshino S.
        • et al.
        Diffusion tensor fiber tracking in patients with central post-stroke pain; correlation with efficacy of repetitive transcranial magnetic stimulation.
        Pain. 2008; 140: 509-518https://doi.org/10.1016/j.pain.2008.10.009
        • Emara T.
        • El Nahas N.
        • Elkader H.A.
        • Ashour S.
        • El Etrebi A.
        MRI can predict the response to therapeutic repetitive transcranial magnetic stimulation (rTMS) in stroke patients.
        J Vasc Interv Neurol. 2009; 2: 163-168
        • Halko M.A.
        • Datta A.
        • Plow E.B.
        • Scaturro J.
        • Bikson M.
        • Merabet L.B.
        Neuroplastic changes following rehabilitative training correlate with regional electrical field induced with tDCS.
        Neuroimage. 2011; 57: 885-891https://doi.org/10.1016/j.neuroimage.2011.05.026
        • Brodie S.M.
        • Borich M.R.
        • Boyd L.A.
        Impact of 5-Hz rTMS over the primary sensory cortex is related to white matter volume in individuals with chronic stroke.
        Eur J Neurosci. 2014; 40: 3405-3412https://doi.org/10.1111/ejn.12717
        • Lai C.-J.
        • Wang C.-P.
        • Tsai P.-Y.
        • Chan R.-C.
        • Lin S.-H.
        • Lin F.-G.
        • et al.
        Corticospinal integrity and motor impairment predict outcomes after excitatory repetitive transcranial magnetic stimulation: a preliminary study.
        Arch Phys Med Rehabil. 2015; 96: 69-75https://doi.org/10.1016/j.apmr.2014.08.014
        • Pavlova E.L.
        • Lindberg P.
        • Khan A.
        • Ruschkowski S.
        • Nitsche M.A.
        • Borg J.
        Transcranial direct current stimulation combined with visuo-motor training as treatment for chronic stroke patients.
        Restor Neurol Neurosci. 2017; 35: 307-317https://doi.org/10.3233/RNN-160706
        • Mane R.
        • Chew E.
        • Phua K.S.
        • Ang K.K.
        • Robinson N.
        • Vinod A.P.
        • et al.
        Prognostic and monitory EEG-biomarkers for BCI upper-limb stroke rehabilitation.
        IEEE Trans Neural Syst Rehabil Eng Publ IEEE Eng Med Biol Soc. 2019; 27: 1654-1664https://doi.org/10.1109/TNSRE.2019.2924742
        • Tamashiro H.
        • Kinoshita S.
        • Okamoto T.
        • Urushidani N.
        • Abo M.
        Effect of baseline brain activity on response to low-frequency rTMS/intensive occupational therapy in poststroke patients with upper limb hemiparesis: a near-infrared spectroscopy study.
        Int J Neurosci. 2019; 129: 337-343https://doi.org/10.1080/00207454.2018.1536053
        • Baltar A.
        • Piscitelli D.
        • Marques D.
        • Shirahige L.
        • Monte-Silva K.
        Baseline motor impairment predicts transcranial direct current stimulation combined with physical therapy-induced improvement in individuals with chronic stroke.
        Neural Plast. 2020; 2020: 8859394https://doi.org/10.1155/2020/8859394