Full length article| Volume 15, ISSUE 3, P881-888, May 01, 2022

# Perceived timing of cutaneous vibration and intracortical microstimulation of human somatosensory cortex

Open AccessPublished:May 26, 2022

## Highlights

• Vibration is perceived by a human faster than touch evoked by ICMS.
• Vibration and ICMS are perceived simultaneously when delivered simultaneously.
• High-intensity ICMS may be necessary for functionally beneficial BCIs.

## Abstract

### Background

Intracortical microstimulation (ICMS) of somatosensory cortex can partially restore the sense of touch. Though ICMS bypasses much of the neuraxis, prior studies have found that conscious detection of touch elicited by ICMS lags behind the detection of cutaneous vibration. These findings may have been influenced by mismatched stimulus intensities, which can impact temporal perception.

### Objective

Evaluate the relative latency at which intensity-matched vibration and ICMS are perceived by a human participant.

### Methods

One person implanted with microelectrode arrays in somatosensory cortex performed reaction time and temporal order judgment (TOJ) tasks. To measure reaction time, the participant reported when he perceived vibration or ICMS. In the TOJ task, vibration and ICMS were sequentially presented and the participant reported which stimulus occurred first. To verify that the participant could distinguish between stimuli, he also performed a modality discrimination task, in which he indicated if he felt vibration, ICMS, or both.

### Results

When vibration was matched in perceived intensity to high-amplitude ICMS, vibration was perceived, on average, 48 ms faster than ICMS. However, in the TOJ task, both sensations arose at comparable latencies, with points of subjective simultaneity not significantly different from zero. The participant could discriminate between tactile modalities above chance level but was more inclined to report feeling vibration than ICMS.

### Conclusions

The latencies of ICMS-evoked percepts are slower than their mechanical counterparts. However, differences in latencies are small, particularly when stimuli are matched for intensity, implying that ICMS-based somatosensory feedback is rapid enough to be effective in neuroprosthetic applications.

## 1. Introduction

Manual touch plays a critical role in object manipulation and performing activities of daily living [
• Johansson R.S.
• Flanagan J.R.
Coding and use of tactile signals from the fingertips in object manipulation tasks.
]. One approach to restoring touch to individuals with somatosensory deficits is to electrically activate neurons in the hand representation of somatosensory cortex [
• Flesher S.N.
• et al.
Intracortical microstimulation of human somatosensory cortex.
,
• Salas M.A.
• et al.
Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation.
,
• Fifer M.S.
• et al.
Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury.
]. These artificial tactile signals can be used to support and enhance the use of a prosthetic hand [
• Flesher S.N.
• et al.
A brain-computer interface that evokes tactile sensations improves robotic arm control.
]. Precise timing of tactile feedback is critical for enabling rapid dexterous object interactions [
• Goebl W.
• Palmer C.
Tactile feedback and timing accuracy in piano performance.
,
• Rohde Marieke E.
• Paolo Di
,
• Chase R.A.
• Rapin I.
• Gilden L.
• Sutton S.
• Guilfoyle G.
], which raises a key question as to whether intracortical microstimulation (ICMS) can be perceived and/or integrated with a speed resembling that of natural somatosensation. Indeed, lags in ICMS-evoked percepts could impair the utility of sensory feedback [
• Fukuda K.
• Hiraki K.
Rubber hand illusion under delayed visual feedback.
,
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
] .
Table 1Temporal order judgment task numerical results. The mean ± standard deviation of the point of subjective simultaneity (PSS) and just noticeable difference (JND) was calculated from the bootstrapped values. Positive PSS values indicate that ICMS needed to occur before vibration in order to be perceived as maximally simultaneous, and negative values indicate that vibration needed to occur first. After Bonferroni correction, none of the PSS values were statistically significantly different from zero. RR = right ring finger, RR = right thumb, LT = left thumb.
Spatially offset vibration and ICMSCo-located vibration and ICMS
PSS
Vibrate RR-12 ± 25 ms-54 ± 24 ms
Vibrate RT-57 ± 29 ms+50 ± 34 ms
Vibrate LT-50 ± 26 ms+13 ± 34 ms
JND
Vibrate RR155 ± 25 ms133 ± 23 ms
Vibrate RT176 ± 28 ms230 ± 40 ms
Vibrate LT152 ± 27 ms216 ± 37 ms
The temporal perception of artificial touch has been studied with peripheral nerve stimulation in humans [
• Christie B.P.
• Graczyk E.L.
• Charkhkar H.
• Tyler D.J.
• Triolo R.J.
Visuotactile synchrony of stimulation-induced sensation and natural somatosensation.
], cortical surface stimulation through electrocorticographic electrodes in humans [
• Caldwell D.J.
• et al.
Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans.
], and ICMS in non-human primates (NHPs) [
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
,
• Godlove J.M.
• Whaite E.O.
• Batista A.P.
Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control.
]. Despite differences in the mechanisms of neural activation, the temporal processing of sensations evoked by peripheral nerve stimulation was not significantly different than the timing of mechanical stimulation of the skin [
• Christie B.P.
• Graczyk E.L.
• Charkhkar H.
• Tyler D.J.
• Triolo R.J.
Visuotactile synchrony of stimulation-induced sensation and natural somatosensation.
]. On the other hand, though ICMS bypasses multiple stages of peripheral processing, sensations evoked via stimulation of somatosensory cortex are typically found to be slower to emerge than sensations evoked via mechanical skin stimulation. This difference in latencies could be explained by mismatched intensities between ICMS and mechanical stimulation; indeed, intensity has a demonstrated effect on perceptual latency [
• Jaśkowski P.
Chapter 11 Reaction time and temporal-order judgment as measures of perceptual latency: the problem of dissociations.
,
• Neumann O.
• Niepel M.
Timing of perception and perception of time.
,
• Sanford A.J.
Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time.
,
• Shore D.I.
• Spence C.
Prior entry.
,
• Smith W.F.
The relative quickness of visual and auditory perception.
,
• Peon A.R.
• Prattichizzo D.
Reaction times to constraint violation in haptics: comparing vibration, visual and audio stimuli.
] as well as processing rate in sensory pathways [
• Nissen M.J.
Stimulus intensity and information processing.
]: the more intense a stimulus is, the faster it is processed. In NHPs, ICMS latencies were able to match mechanical stimulation latencies only when high currents were delivered through at least four electrodes simultaneously [
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
].
The objective of the present study was to characterize the latency at which ICMS-evoked sensations were consciously experienced, and to compare the latency of artificial touch to its mechanically-evoked and intensity-matched counterpart in humans. To this end, a human participant implanted with microelectrode arrays in somatosensory cortex performed two sensory tasks: a reaction time task and a temporal order judgment (TOJ) task. In the reaction time task, the participant was presented with a cutaneous vibration or an ICMS pulse train and provided a speeded verbal report when he perceived it. In the TOJ task, the participant was sequentially presented with a pair of stimuli – vibration followed by an ICMS pulse train or vice versa – and judged which of the two occurred first. The participant also performed a modality discrimination task to verify that he could discriminate ICMS from vibration, a necessary precondition to perform the TOJ task.

## 2. Methods

### 2.1 Human participant

The following experiments were conducted with a 50-year-old male with C5 (sensory), C6 (motor) ASIA B tetraplegia. He had retained intact somatosensation in his fingertips as indicated by clinical reports. This study was conducted under Investigational Device Exemption (IDE, G170010) by the Food and Drug Administration (FDA) for the purpose of evaluating bilateral sensory and motor capabilities of intracortical microelectrode arrays. The study protocol was approved by the FDA, the Johns Hopkins Institutional Review Board (JH IRB) and the NIWC Human Research Protection Office, and is a registered clinical trial (NCT03161067). The participant gave his written informed consent prior to participation in research-related activities.

### 2.2 Cutaneous vibration of the hand

A vibratory stimulus (“natural touch”) was delivered on the hand using a miniature electromechanical tactor (type C-2, Engineering Acoustics, Inc.; Casselberry, FL, USA). The tactor was controlled by sinusoidal signals that were digitally generated in MATLAB (MathWorks, Inc.; Natick, MA, USA). Output signals passed through the computer's audio port and were amplified (Pyle, PTA4 Stereo Power Amplifier). Vibratory inputs were delivered at 300 Hz, which was assumed to primarily activate Pacinian fibers in the skin [
• Abraira V.E.
• Ginty D.D.
The sensory neurons of touch.
,
• McGlone F.
• Reilly D.
The cutaneous sensory system.
]. We did not anticipate the vibration frequency to impact our experimental results because Pacinian and non-Pacinian afferent fibers have similar diameters and therefore similar conduction velocities [
• Abraira V.E.
• Ginty D.D.
The sensory neurons of touch.
]. The displacement of the tactor could be varied to change the indentation depth into the skin. The tactor was fastened to the hand using medical tape. The participant reported that he could not hear any noise from the tactor so he did not wear noise-cancelling headphones during experiments.

### 2.3 Intracortical microstimulation of somatosensory cortex to elicit tactile percepts in the hand

This study involved a human participant who was chronically implanted with microelectrode arrays in the bilateral primary motor cortices and Area 1 of the somatosensory cortices (see Fig. 1a, and McMullen et al. and Fifer et al. for additional details on these implants [
• Fifer M.S.
• et al.
Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury.
,
• McMullen D.P.
• et al.
Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: technical note.
]). In the present study, only the stimulating electrodes implanted in somatosensory cortex were used. Two of the arrays were implanted in the left somatosensory cortex and one was in the right hemisphere. To access an external stimulator and software, the arrays were wired to skull-fixed transcutaneous metal pedestals. A CereStim R96 (Blackrock Neurotech; Salt Lake City, UT, USA) was used to deliver electrical stimulation to the electrodes. The electrical stimulation waveforms were biphasic, charge-balanced, cathodic-first pulses and were grounded to the metal pedestal. Pulse frequency was set to 100 Hz, total pulse width was 500 μs (200 μs for each phase with a 100 μs interphase delay), and pulse amplitude varied between 30 and 80 μA. Modulating the pulse frequency of ICMS can also modulate the perceived intensity of ICMS, however the direction of that change varies across electrodes [
• Hughes C.L.
• Flesher S.N.
• Weiss J.M.
• Boninger M.
• Collinger J.L.
• Gaunt R.A.
Perception of microstimulation frequency in human somatosensory cortex.
], which is why we modulated amplitude instead of frequency. Stimulation parameters stayed within safety limits to minimize the risk of tissue and/or electrode damage [
• Rajan A.T.
• et al.
The effects of chronic intracortical microstimulation on neural tissue and fine motor behavior.
]. Stimulation pulse trains were controlled via MATLAB scripts that sent commands to a custom C++-based stimulator interface application.
For each site on the hand, ICMS was delivered through a pair of electrodes that had similar projected fields on the participant's hand. This was done to increase the detectability of ICMS, making it easier and more reliable for the participant to perceive. Projected fields were estimated by stimulating through each individual electrode and asking the participant to verbally report the regions of the hand in which he felt a tactile percept. These electrode mappings and the detection thresholds of pairs of electrodes are more thoroughly discussed in Fifer et al. [
• Fifer M.S.
• et al.
Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury.
]. Both electrodes in a pair were always part of the same array, but the electrode pairs were not always the same across all three tasks (Supplemental Fig. 1).

### 2.4 Intensity matching between vibration and ICMS

Because temporal perception is strongly affected by variations in stimulus intensity [
• Jaśkowski P.
Chapter 11 Reaction time and temporal-order judgment as measures of perceptual latency: the problem of dissociations.
,
• Neumann O.
• Niepel M.
Timing of perception and perception of time.
,
• Sanford A.J.
Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time.
,
• Shore D.I.
• Spence C.
Prior entry.
,
• Smith W.F.
The relative quickness of visual and auditory perception.
,
• Verleger R.
Attentional bias toward low-intensity stimuli: an explanation for the intensity dissociation between reaction time and temporal order judgment?.
], we performed perceptual intensity matching between vibration and ICMS on each hand site at the start of each experimental session for all three experimental tasks. Two above-threshold ICMS intensities (one lower, one higher) were chosen for the reaction time task. The lower ICMS amplitude value was generally 1–4.5 dB above the last measured detection threshold for the chosen electrode pair, typically 30 or 45 μA. The higher ICMS amplitude was always chosen to be 80 μA, which was generally 6–9.4 dB above the last measured detection threshold for the chosen electrode pair. An amplitude of 80 μA was used for the TOJ and modality discrimination tasks.
Vibration intensity was matched to a given ICMS intensity using a modified adaptive 2-down, 1-up staircase paradigm with a two-alternative forced choice (2-AFC) presentation of vibration and ICMS, in which the participant chose which stimulus was perceived as being more intense. To enable faster identification of perceived matched magnitudes, we modified the adaptive procedure to allow the participant to report if the perceived intensities were the same on any given trial. When the participant perceived both vibration and ICMS to be of the same magnitude, the adaptive staircase paradigm was stopped. Vibration intensity was modulated by changing the indentation amplitude with a constant step size, which translated to approximately 0.8–1.6 dB of the vibration detection threshold. Each site on the hand had a slightly different vibration detection threshold, so the amplitude step size was chosen accordingly.
Vibration and ICMS levels were converted to dB using the following relation,
$Equation 1.$
Equation 1

in which A is the stimulus amplitude level and T is the stimulation detection threshold for the respective stimulation modality at that location. This conversion was done so that the reported stimulation levels could be more easily compared to each other by directly relating them to the detection threshold for each stimulation modality type (i.e., vibration or ICMS).

The participant performed a reaction time test for vibratory stimuli, ICMS pulse trains, and visual stimuli (Fig. 1b). The participant was instructed to verbally respond with the word “now” into a microphone when he felt a tactile stimulus or observed the visual stimulus (which served as a control condition). Verbal responses were captured using a microphone connected to an H6 audio recorder (Zoom Corporation; Hauppauge, NY, USA) with the raw audio outputted directly to the analog input on a Neuroport Neural Signal Processor (Blackrock Neurotech). The audio waveform was recorded at 30 kHz by the Neuroport, which was also recording the stimulation waveform at 2 kHz. The stimulation and audio waveforms were recorded on the same Neuroport and were aligned using global timestamps. The audio signal was down-sampled to 2 kHz to match the stimulation sampling rate. Reaction time was measured as the time difference between stimulation onset and verbal response onset. Verbal response onset was defined as when the recorded audio signal exceeded a manually-defined threshold, which was approximately six standard deviations above baseline. We chose to collect the participant's responses verbally rather than using a manual button press test due to limitations in the participant's hand dexterity resulting from his spinal cord injury. The audio recordings were screened to ensure that they did not include any accidental vocalizations.
Each stimulus lasted for 500 ms with a random delay of 3–7 s between each trial. Reaction times that fell below 100 ms or above 1000 ms were considered physiologically implausible and excluded from the analysis [
• Miller J.
• Franz V.
• Ulrich R.
Effects of auditory stimulus intensity on response force in simple, go/no-go, and choice RT tasks.
,
• Włodarczyk D.
• Jaśkowski P.
• Nowik A.
Influence of sleep deprivation and auditory intensity on reaction time and response force.
]. There were 54 trials with responses outside of this range (18 vibration, 17 ICMS, and 19 visual). Experiments were performed in blocks of 10 trials to minimize the occurrence of perceptual adaptation [
• Hughes C.L.
• Flesher S.N.
• Gaunt R.A.
Effects of stimulus pulse rate on somatosensory adaptation in the human cortex.
]. After removing outlier responses, a total of 287 trials were collected for visual stimuli, 282 trials were collected for vibration, and 279 trials were collected for ICMS. Data were collected across eight sessions, approximately once a week, over the course of three months.
Only one stimulus type (vibration, ICMS, or vision) was presented at a time. During the visual reaction time test, the participant viewed a monitor and responded when the screen changed colors. For the vibration and ICMS tests, we tested five different sites across both hands, at two different intensity levels per site (Supplemental Table 1). The participant focused his gaze on his hand while receiving vibration or ICMS.

### 2.6 Temporal order judgment task

To evaluate the temporal synchrony of cutaneous vibration and ICMS, we conducted a two-alternative forced choice TOJ task (Fig. 1c). In this task, vibration and ICMS were sequentially delivered and the participant was asked which stimulus he perceived as occurring first. The stimuli were separated by a stimulus onset asynchrony (SOA) value of 0, ±50, ±100, ±200, ±300, or ±500 ms. Positive SOA values indicate that ICMS occurred before vibration, and negative values indicate that vibration occurred first. Any hardware latencies were accounted for such that, if commanded to be delivered simultaneously, the onset of vibration and ICMS were aligned within 1 ms. The start of each trial was indicated by a brief audio cue, and each stimulus lasted for 200 ms. We chose three sites on the hands to apply vibration and subsequently picked stimulating electrodes that elicited ICMS percepts in the same region or spatially offset regions of the hand, depending on the experimental condition (Supplemental Table 1). The order of application of the 11 SOA values was randomized and each SOA was tested 20 times per site. In one instance, in which vibration and ICMS were delivered to the right thumb, we tested each SOA 36 times; when we fit the logistic curve to the raw experimental data after just 20 repetitions, the R2 was only 0.63, and we anticipated that additional trials would smooth out any initial noise in the raw data. Each SOA was tested one time per site per experimental block; blocks were kept short to minimize perceptual adaptation. Data were collected in 18 sessions over the course of eight months.
To compare the TOJ results across conditions, we used a parametric bootstrap method with 1000 replications [
• Azañón E.
• Soto-Faraco S.
Alleviating the ‘crossed-hands’ deficit by seeing uncrossed rubber hands.
,
• Efron B.
• Tibshirani R.J.
An introduction to the bootstrap.
,
• Kanai R.
• Sheth B.R.
• Shimojo S.
Stopping the motion and sleuthing the flash-lag effect: spatial uncertainty is the key to perceptual mislocalization.
]. This involved generating a synthetic dataset by sampling from a binomial distribution $B(n,p)$, in which n = 20 trials for each SOA, and p = the probability that the participant reported that ICMS came first based on a sigmoidal curve (Equation (2)) fit to the raw experimental data. Sigmoidal curves were then fit to each synthetic data set to extract two outcome measures: the point of subjective simultaneity (PSS) and just noticeable difference (JND) (Supplemental Fig. 2a). The PSS equaled the SOA that corresponded to when the participant reported that he felt ICMS first in 50% of trials. In other words, the PSS represented the temporal offset that resulted in vibration and ICMS being perceived as maximally simultaneous. The JND was equal to the difference in SOA between the 25% and 75% points divided by two, and represents the smallest temporal interval that the participant could reliably detect [
• Keetels M.
• Vroomen J.
Perception of synchrony between the senses.
].
$Equation 2.$
Equation 2

Performing the TOJ task with spatially overlapping vibration and ICMS required that the participant be able to distinguish vibration from ICMS. To test this, we delivered vibration alone, ICMS alone, no stimulus, or both vibration and ICMS in the same region of the hand (“multiplexed”). The participant's task was to report which class(es) of stimulus was presented. If both stimuli were presented, he did not have to specify which came first, seeing as they were delivered simultaneously. Two other conditions, in which vibration and ICMS were delivered in the same trial but were temporally offset, were initially included in the experiment but those data were excluded from the analysis because those trials had longer overall durations, which would confound the results.
Similar to the TOJ task, each stimulus in the modality discrimination task lasted for 200 ms. This train duration was intended to be a compromise between being long enough to be easily detected by the participant, but not so long that it expedited the process of perceptual adaptation. In contrast, a 500 ms duration was used for the reaction time tests. This increased duration was chosen to increase the detectability of each stimulus by the participant, making the reaction time task as simple as possible.
Each trial began with an auditory cue followed by a 1 s delay and the stimuli. ICMS amplitude was always 80 μA and the vibration amplitude was matched in perceived intensity using the approach described above. This task was performed at four locations across both hands (Supplemental Table 1), each in different experimental blocks. Each experimental block consisted of 20 trials (five repetitions of each condition). Data were collected over four sessions, approximately once every two weeks over the course of two months. There were a total of 30 trials per condition on the right thumb and right index finger and 15 trials per condition at the other two locations.

### 2.8 Statistical analyses

We performed two-sample two-tailed Wilcoxon rank-sum tests to compare the reaction times of the tactile stimuli (i.e., vibration vs. ICMS) as well as tactile stimuli vs. visual stimuli. To compare low- vs. high-intensity stimuli, we ran paired-sample one-tailed Wilcoxon signed-rank tests, with the hypothesis that more intense stimuli would be perceived more quickly. Data normality was not tested because we used nonparametric statistical tests.
For the TOJ results, we fit a probability density function to the 1000 bootstrapped PSS values and calculated the proportion of samples that fell above zero, which was assumed to be the one-tailed p-value (Supplemental Fig. 2b). A two-tailed test, to evaluate whether the PSS was significantly above or below zero, was performed by doubling the one-tailed p-value. The PSS data did not deviate significantly from normality (Shapiro-Wilk test, p > 0.05). Additionally, to compare the PSS and JND values between the spatially co-located vs. offset conditions, we calculated the proportion of overlap (taken to be the p-value) between the probability density functions of the two conditions (Supplemental Fig. 2c).
To analyze the modality discrimination task results, we ran one-sample binomial tests to determine if the participant's ability to correctly identify a stimulus was better than chance (25%). The statistical tests were one-tailed because we hypothesized that the participant could discriminate between stimuli at an accuracy above chance. Significance levels in all statistical analyses were set to α = 0.05 and Bonferroni corrections were applied for each task. MATLAB was used to perform these analyses.

## 3. Results

First, we examined the speed at which the participant could report feeling vibration and ICMS. Vibratory stimuli and ICMS were presented at two intensities, matched for perceived magnitude. We observed reaction times of 366 ± 106 ms (mean ± standard deviation) for the low-intensity vibrations and 352 ± 111 ms for the high-intensity vibrations (Fig. 2 and Supplemental Fig. 3). The reaction times were faster for the more intense stimuli (p < 0.05), as expected. Reaction times were also consistent with previously reported values from studies in which participants responded using a button press, which ranged from 177 to 400 ms [
• Caldwell D.J.
• et al.
Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans.
,
• Diederich A.
• Colonius H.
Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time.
,
• Lele P.P.
• Sinclair D.C.
• Weddell G.
The reaction time to touch.
], particularly considering that vocal responses tend to be 60–80 ms slower than manual ones [
• Nebes R.D.
Vocal versus manual response as a determinant of age difference in simple reaction time.
]. Reaction times to ICMS pulse trains were 456 ± 143 ms for low-intensity stimuli and 400 ± 140 ms for high-intensity stimuli. Similar to prior studies [
• Jaśkowski P.
Chapter 11 Reaction time and temporal-order judgment as measures of perceptual latency: the problem of dissociations.
,
• Neumann O.
• Niepel M.
Timing of perception and perception of time.
,
• Sanford A.J.
Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time.
,
• Shore D.I.
• Spence C.
Prior entry.
,
• Smith W.F.
The relative quickness of visual and auditory perception.
,
• Peon A.R.
• Prattichizzo D.
Reaction times to constraint violation in haptics: comparing vibration, visual and audio stimuli.
] and our results with vibratory stimuli, the effect of intensity on ICMS reaction time was statistically significant (p < 0.001). Importantly, reaction times to ICMS were slower than reaction times to vibration (p < 0.001 at both low and high intensities).
To further benchmark the participant's reaction times with respect to individuals without sensorimotor deficits, we measured his reaction times to visual stimuli. We found that the mean reaction time of 295 ± 75 ms was comparable to other normally-sighted individuals (around 330 ms [
• Nebes R.D.
Vocal versus manual response as a determinant of age difference in simple reaction time.
]). The participant's reaction time to visual stimuli was significantly faster than his reaction to tactile stimuli (p < 0.001 at either intensity level for either tactile stimulus).

### 3.2 Temporal order judgment task

Next, we examined the participant's ability to judge which of two sequentially presented stimuli – a vibration or an ICMS pulse train – preceded the other. The stimulus onset asynchrony varied from −500 to +500 ms, with negative offsets indicating that vibration preceded ICMS. When vibration and ICMS elicited percepts in spatially offset regions of the hand, the point of subjective simultaneity (PSS) was −12 ± 25 ms for one pair of stimulus locations (vibration on right ring, ICMS on right thumb), −57 ± 29 ms for another (vibration on right thumb, ICMS on right ring), and −50 ± 26 ms for a third (vibration on left thumb, ICMS on left index) (Fig. 3a and Supplemental Fig. 4). While the PSSs were systematically less than zero, none were significantly so, implying that ICMS- and vibration-evoked percepts did not differ significantly in latency.
When vibration- and ICMS-evoked sensations were co-located, PSSs were −54 ± 24 ms for the right ring finger, +50 ± 34 ms for the right thumb, and +13 ± 34 ms for the left thumb (Fig. 3b). Again, PSSs were not significantly different from zero after Bonferroni correction. The PSS values in the spatially offset vs. co-located conditions were also not significantly different (p > 0.05).
Finally, the psychometric functions relating performance to SOA show that the participant was sensitive to changes in timing over hundreds of milliseconds. Indeed, when vibration and ICMS were spatially offset, the just noticeable differences (JNDs) – the smallest interval that the participant could reliably detect – were 155 ± 25 ms, 176 ± 28 ms, and 152 ± 27 ms for the three conditions. When vibration and ICMS were co-located, the JNDs were 133 ± 23 ms, 230 ± 40 ms, and 216 ± 37 ms. The JND values in the spatially offset vs. co-located conditions were not significantly different (p > 0.05).

While the reaction time task revealed a significant difference in the perceptual latencies of vibration and ICMS, the TOJ task did not. One possibility is that the participant had difficulty distinguishing vibration from ICMS. To test this possibility, we delivered a vibratory stimulus, an ICMS pulse train, or both, and asked the participant to judge which of the conditions had been presented. We found that the participant was able to reliably identify the stimulus modalities delivered across the four sites tested (Fig. 4 and Supplemental Fig. 5). When ICMS was presented in isolation, he described the sensation as “pressure” regardless of which electrode was stimulated. “ICMS only” conditions were correctly classified 93% of the time and “vibration only” conditions were classified correctly 86% of the time. When both stimuli were presented simultaneously (the “multiplexed” condition in Fig. 4), the participant's accuracy dropped to 69% correct, but was still significantly higher than chance (p < 0.001). When the participant misclassified trials containing both stimuli, he was most likely (89%) to report perceiving vibration only, despite stimulus levels being matched in perceived intensity.

## 4. Discussion

The goal of this study was to evaluate the relative latency at which intensity-matched artificial and natural touch sensations are perceived. To this end, a human participant implanted with stimulating microelectrodes in his somatosensory cortices performed a reaction time task and a TOJ task with cutaneous vibration and ICMS [
• Fifer M.S.
• et al.
Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury.
,
• McMullen D.P.
• et al.
Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: technical note.
]. Consistent with previous studies [
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
,
• Caldwell D.J.
• et al.
Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans.
,
• Godlove J.M.
• Whaite E.O.
• Batista A.P.
Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control.
], we found that the average reaction time to high-intensity vibration was 48 ms faster than high-intensity tactile percepts elicited by cortical stimulation, even when the two stimuli were matched in perceived intensity. When both vibration and ICMS were delivered at a lower intensity level, the difference in reaction time was even larger: vibration was perceived 90 ms before ICMS, on average. However, in the TOJ task, vibratory and ICMS sensations seemed to arise at comparable latencies, with estimated PSS values which were not significantly different from zero. One possible explanation for the discrepancy between the reaction time and TOJ test results is that when the two stimuli were in close temporal proximity, the participant had difficulty distinguishing one from the other (see below).

### 4.1 Vibration is perceived more quickly than ICMS

In this study, we found that the reaction time to vibration was faster than that of ICMS. This result refuted our original prediction, which was that the reaction time to artificial touch would be faster than vibration if the stimuli were intensity matched, but is consistent with previous findings [
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
,
• Caldwell D.J.
• et al.
Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans.
,
• Godlove J.M.
• Whaite E.O.
• Batista A.P.
Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control.
]. One possible explanation for the delayed perception of ICMS is that ICMS activates somatosensory cortex neurons in an unnatural way [
• Godlove J.M.
• Whaite E.O.
• Batista A.P.
Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control.
]. Indeed, ICMS results in the simultaneous activity of hundreds or thousands of neurons around the stimulating electrode in a way that would never naturally occur [
• Tehovnik E.J.
Electrical stimulation of neural tissue to evoke behavioral responses.
]; the participant reported that ICMS felt different than normal pressure and vibration. Another possible explanation for the slower latency of ICMS-evoked percepts is that ICMS bypasses several processing stages, including the cuneate nucleus, the thalamus, and Brodmann's Area 3b [
• Delhaye B.P.
• Long K.H.
• Bensmaia S.J.
Neural basis of touch and proprioception in primate cortex.
]. In natural touch, the concerted activation of these parallel pathways may facilitate downstream processing, thereby leading to more rapidly evolving percepts. To disentangle these different possibilities will require additional experiments involving the electrical stimulation of different structures along the somatosensory neuraxis.

### 4.2 Disparity between reaction time and TOJ results

In contrast to the reaction time results, results from the TOJ task suggested that mechanically- and electrically-evoked sensations emerge with approximately the same latency. The participant's difficulty discriminating between stimulus modalities may have hindered his performance on the TOJ task. Though the participant performed well above chance in the modality discrimination task, the trials in which both vibration and ICMS were presented at the same time were still challenging (accuracy = 69%). When the participant misclassified a stimulus, he often reported feeling vibration only, suggesting a dominance of vibration over ICMS despite our attempts to match the perceived magnitudes of both stimuli.
Note, however, that discrepancies between reaction time and TOJ tasks have been previously observed [
• Miller J.
• Schwarz W.
Dissociations between reaction times and temporal order judgments: a diffusion model approach.
,
• Rutschmann J.
Perception of temporal order of stimuli differing in sense mode and simple reaction time.
] and may reflect differences in task demands. Specifically, reaction time judgments are speeded whereas TOJ judgments are not. Accordingly, the nervous system may have time to resolve minor differences in sensory latency by the time the participant has to make a TOJ judgment. The reaction time test also involved unimodal stimuli whereas the TOJ task involved bimodal stimuli. Another possibility is that the participant was biased towards reporting ICMS first because the evoked percept did not feel natural, which required greater concentration to detect and interpret. This type of selective attentional bias can reduce detection time in a TOJ task [
• Scharlau I.
• Neumann O.
Perceptual latency priming by masked and unmasked stimuli: evidence for an attentional interpretation.
,
• Stelmach L.B.
• Herdman C.M.
Directed attention and perception of temporal order.
,
• Sternberg S.
• Knoll R.L.
The perception of temporal order: fundamental issues and a general model.
].

### 4.3 Limitations

Although we used previously validated experimental techniques, this study had a number of limitations. Our findings could become more generalizable if they were repeated in a larger sample size. Furthermore, though we matched the intensity between vibration and ICMS pairings, we did not match the intensities across pairings, which likely accounts for some of the across-site variation (Supplemental Figs. 3–5). It is also possible that because of the participant's spinal cord injury, he had sensory deficits in his hands that could not be detected via clinical reports or personal awareness, which could have impacted the detectability of vibratory stimuli and caused differences in results across tested hand sites. An interesting possibility for future work would be to systemically characterize the effect of other ICMS parameters, such as pulse frequency and pulse train length, on the temporal perception of ICMS.

### 4.4 Implications for brain-computer interfaces

Prior studies have demonstrated that somatosensory feedback improves performance on dexterous motor tasks [
• Johansson R.S.
• Flanagan J.R.
Coding and use of tactile signals from the fingertips in object manipulation tasks.
,
• Monzée J.
• Lamarre Y.
• Smith A.M.
The effects of digital anesthesia on force control using a precision grip.
] while minimizing reliance on visual feedback [
• Ghez C.
• Gordon J.
• Ghilardi M.F.
Impairments of reaching movements in patients without proprioception. II. Effects of visual information on accuracy.
]. Touch feedback provided via ICMS has recently been demonstrated to improve reach-to-grasp task completion times for a person with spinal cord injury [
• Flesher S.N.
• et al.
A brain-computer interface that evokes tactile sensations improves robotic arm control.
]. Despite our observation that the perception of ICMS is slower than vibration, NHP studies have demonstrated that ICMS perception can be accelerated by increasing stimulating current across multiple electrodes [
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
]. Additionally, the nervous system maintains multisensory synchrony by correcting for short time lags [
• Keetels M.
• Vroomen J.
Perception of synchrony between the senses.
], such that any remaining difference in latency between natural and artificial touch can likely be compensated for.

## Disclaimer

The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

## CRediT authorship contribution statement

Breanne Christie: Conceptualization, Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. Luke E. Osborn: Conceptualization, Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. David P. McMullen: Conceptualization, Methodology, Investigation, Formal analysis, Writing – review & editing. Ambarish S. Pawar: Investigation, Writing – review & editing. Tessy M. Thomas: Investigation, Writing – review & editing. Sliman J. Bensmaia: Conceptualization, Methodology, Formal analysis, Writing – review & editing, Visualization. Pablo A. Celnik: Conceptualization, Resources, Writing – review & editing, Supervision, Project administration, Funding acquisition. Matthew S. Fifer: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing – review & editing, Resources, Supervision, Project administration, Funding acquisition. Francesco V. Tenore: Conceptualization, Methodology, Formal analysis, Investigation, Writing – review & editing, Resources, Supervision, Project administration, Funding acquisition.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgements

The authors thank Pawel Kudela, Brock Wester, Manuel Anaya, and the research participant. This work was made possible, in part, through financial support from Defense Advanced Research Projects Agency (DARPA) under the Neurally Enhanced Operations program (contract number HR001120C0120 ) and the Johns Hopkins University Applied Physics Laboratory . In addition, this work was supported with the resources and use of facilities at the Johns Hopkins Hospital.

## Appendix A. Supplementary data

• Supplemental Table 1

The experimental conditions for all three tasks. The reaction time task and some conditions of the modality discrimination task presented only one stimulus per trial; therefore, we wrote “N/A” in the “Stimulus B” column. Ihigh and Ilow signify that the test was performed once with a high-intensity stimulus and once with a low-intensity stimulus.

## References

• Johansson R.S.
• Flanagan J.R.
Coding and use of tactile signals from the fingertips in object manipulation tasks.
Nat Rev Neurosci. May 2009; 10: 345-359https://doi.org/10.1038/nrn2621
• Flesher S.N.
• et al.
Intracortical microstimulation of human somatosensory cortex.
Sci Translat Med. 2016; : 1-11https://doi.org/10.1126/scitranslmed.aaf8083
• Salas M.A.
• et al.
Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation.
Elife. 2018; https://doi.org/10.7554/eLife.32904
• Fifer M.S.
• et al.
Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury.
Neurology. Dec. 2021; (10.1212/WNL.0000000000013173)https://doi.org/10.1212/WNL.0000000000013173
• Flesher S.N.
• et al.
A brain-computer interface that evokes tactile sensations improves robotic arm control.
Science (1979). May 2021; 372: 831-836https://doi.org/10.1126/science.abd0380
• Goebl W.
• Palmer C.
Tactile feedback and timing accuracy in piano performance.
Exp Brain Res. Apr. 2008; 186: 471-479https://doi.org/10.1007/S00221-007-1252-1/FIGURES/5
• Rohde Marieke E.
• Paolo Di
in: Advances in artificial life. 2007: 193-202
• Chase R.A.
• Rapin I.
• Gilden L.
• Sutton S.
• Guilfoyle G.
II Sensory feedback influences on keytapping motor tasks. vol. 13. Sep. 2008: 153-167https://doi.org/10.1080/17470216108416488 (no. 3)
• Fukuda K.
• Hiraki K.
Rubber hand illusion under delayed visual feedback.
PLoS One. Jul. 2009; 4: e6185https://doi.org/10.1371/journal.pone.0006185
• Sombeck J.T.
• Miller L.E.
Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback.
2020https://doi.org/10.1088/1741-2552/ab5cf3
• Christie B.P.
• Graczyk E.L.
• Charkhkar H.
• Tyler D.J.
• Triolo R.J.
Visuotactile synchrony of stimulation-induced sensation and natural somatosensation.
J Neural Eng. 2019; 16https://doi.org/10.1088/1741-2552/ab154c
• Caldwell D.J.
• et al.
Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans.
Sci Rep. 2019; https://doi.org/10.1038/s41598-019-38619-2
• Godlove J.M.
• Whaite E.O.
• Batista A.P.
Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control.
J Neural Eng. 2014; https://doi.org/10.1088/1741-2560/11/4/046025
• Jaśkowski P.
Chapter 11 Reaction time and temporal-order judgment as measures of perceptual latency: the problem of dissociations.
in: Aschersleben G. Bachmann T. Müsseler J. Advances in psychology. vol. 129. North-Holland, 1999: 265-282https://doi.org/10.1016/S0166-4115(99)80026-0
• Neumann O.
• Niepel M.
Timing of perception and perception of time.
in: Kaernbach C. Schroger E. Müller H. Psychophysics beyond sensation: laws and invariants of human cognition. Psychology Press, 2004: 245-269 ([Online]. Available:)
• Sanford A.J.
Effects of changes in the intensity of white noise on simultaneity judgements and simple reaction time.
Q J Exp Psychol. Aug. 1971; 23: 296-303https://doi.org/10.1080/14640746908401824
• Shore D.I.
• Spence C.
Prior entry.
in: Neurobiology of attention. Elsevier, 2005: 89-95https://doi.org/10.1016/B978-012375731-9/50019-7
• Smith W.F.
The relative quickness of visual and auditory perception.
J Exp Psychol. Apr. 1933; 16 ([Online]. Available:): 239-257
• Peon A.R.
• Prattichizzo D.
Reaction times to constraint violation in haptics: comparing vibration, visual and audio stimuli.
in: 2013 world haptics conference (WHC). 2013: 657-661
• Nissen M.J.
Stimulus intensity and information processing.
Percept Psychophys. 1977; 22: 338-352
• Abraira V.E.
• Ginty D.D.
The sensory neurons of touch.
Neuron. 2013; https://doi.org/10.1016/j.neuron.2013.07.051
• McGlone F.
• Reilly D.
The cutaneous sensory system.
Neurosci Biobehav Rev. 2010; https://doi.org/10.1016/j.neubiorev.2009.08.004
• Abraira V.E.
• Ginty D.D.
The sensory neurons of touch.
Neuron. 2013; 79: 618-639https://doi.org/10.1016/j.neuron.2013.07.051
• McMullen D.P.
• et al.
Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: technical note.
J Neurosurg. Mar. 2021; 1 (aop): 1-8https://doi.org/10.3171/2020.9.JNS202675
• Hughes C.L.
• Flesher S.N.
• Weiss J.M.
• Boninger M.
• Collinger J.L.
• Gaunt R.A.
Perception of microstimulation frequency in human somatosensory cortex.
Elife. 2021; 10e65128
• Rajan A.T.
• et al.
The effects of chronic intracortical microstimulation on neural tissue and fine motor behavior.
J Neural Eng. 2015; https://doi.org/10.1088/1741-2560/12/6/066018
• Verleger R.
Attentional bias toward low-intensity stimuli: an explanation for the intensity dissociation between reaction time and temporal order judgment?.
Conscious Cognit. 2000; https://doi.org/10.1006/ccog.2000.0461
• Miller J.
• Franz V.
• Ulrich R.
Effects of auditory stimulus intensity on response force in simple, go/no-go, and choice RT tasks.
Percept Psychophys. 1999; https://doi.org/10.3758/BF03211952
• Włodarczyk D.
• Jaśkowski P.
• Nowik A.
Influence of sleep deprivation and auditory intensity on reaction time and response force.
Percept Mot Skills. 2002; https://doi.org/10.2466/pms.2002.94.3c.1101
• Hughes C.L.
• Flesher S.N.
• Gaunt R.A.
Effects of stimulus pulse rate on somatosensory adaptation in the human cortex.
bioRxiv. Jan. 2021; (2021.12.04.471210)https://doi.org/10.1101/2021.12.04.471210
• Azañón E.
• Soto-Faraco S.
Alleviating the ‘crossed-hands’ deficit by seeing uncrossed rubber hands.
Exp Brain Res. 2007; https://doi.org/10.1007/s00221-007-1011-3
• Efron B.
• Tibshirani R.J.
An introduction to the bootstrap.
1993https://doi.org/10.1007/978-1-4899-4541-9
• Kanai R.
• Sheth B.R.
• Shimojo S.
Stopping the motion and sleuthing the flash-lag effect: spatial uncertainty is the key to perceptual mislocalization.
Vis Res. 2004; https://doi.org/10.1016/j.visres.2003.10.028
• Keetels M.
• Vroomen J.
Perception of synchrony between the senses.
in: Murray M.M. Wallace M.T. The neural bases of multisensory processes. CRC Press/Taylor & Francis, Boca Raton (FL2012 ([Online]. Available:)
http://www.ncbi.nlm.nih.gov/books/NBK92837/
Date accessed: April 20, 2017
• Diederich A.
• Colonius H.
Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time.
Percept Psychophys. 2004; https://doi.org/10.3758/BF03195006
• Lele P.P.
• Sinclair D.C.
• Weddell G.
The reaction time to touch.
J Physiol. 1954; https://doi.org/10.1113/jphysiol.1954.sp005042
• Nebes R.D.
Vocal versus manual response as a determinant of age difference in simple reaction time.
J Gerontol. 1978; https://doi.org/10.1093/geronj/33.6.884
• Tehovnik E.J.
Electrical stimulation of neural tissue to evoke behavioral responses.
J Neurosci Methods. Mar. 1996; 65: 1-17https://doi.org/10.1016/0165-0270(95)00131-X
• Delhaye B.P.
• Long K.H.
• Bensmaia S.J.
Neural basis of touch and proprioception in primate cortex.
Compr Physiol. Oct. 2018; 8: 1575-1602https://doi.org/10.1002/CPHY.C170033
• Miller J.
• Schwarz W.
Dissociations between reaction times and temporal order judgments: a diffusion model approach.
J Exp Psychol Hum Percept Perform. 2006; https://doi.org/10.1037/0096-1523.32.2.394
• Rutschmann J.
Perception of temporal order of stimuli differing in sense mode and simple reaction time.
Percept Mot Skills. 1964; https://doi.org/10.2466/pms.1964.18.2.345
• Scharlau I.
• Neumann O.
Perceptual latency priming by masked and unmasked stimuli: evidence for an attentional interpretation.
Psychol Res. 2003; https://doi.org/10.1007/s00426-002-0116-3
• Stelmach L.B.
• Herdman C.M.
Directed attention and perception of temporal order.
J Exp Psychol Hum Percept Perform. 1991; https://doi.org/10.1037/0096-1523.17.2.539
• Sternberg S.
• Knoll R.L.
The perception of temporal order: fundamental issues and a general model.
Atten Perform. 1973; IV ([Online]. Available:): 629-685
• Johansson R.S.
• Flanagan J.R.
Coding and use of tactile signals from the fingertips in object manipulation tasks.
Nat Rev Neurosci. 2009; https://doi.org/10.1038/nrn2621
• Monzée J.
• Lamarre Y.
• Smith A.M.
The effects of digital anesthesia on force control using a precision grip.
J Neurophysiol. 2003; https://doi.org/10.1152/jn.00434.2001
• Ghez C.
• Gordon J.
• Ghilardi M.F.
Impairments of reaching movements in patients without proprioception. II. Effects of visual information on accuracy.
J Neurophysiol. 1995; https://doi.org/10.1152/jn.1995.73.1.361