FingerSight: A Vibrotactile Wearable Ring for Assistance With Locating and Reaching Objects in Peripersonal Space.

This paper describes a prototype guidance system, "FingerSight," to help people without vision locate and reach to objects in peripersonal space. It consists of four evenly spaced tactors embedded into a ring worn on the index finger, with a small camera mounted on top. Computer-vision analysis of the camera image controls vibrotactile feedback, leading users to move their hand to near targets. Two experiments tested the functionality of the prototype system. The first found that participants could discriminate between five different vibrotactile sites (four individual tactors and all simultaneously) with a mean accuracy of 88.8% after initial training. In the second experiment, participants were blindfolded and instructed to move their hand wearing the device to one of four locations within arm's reach, while hand trajectories were tracked. The tactors were controlled using two different strategies: (1) repeatedly signal axis with largest error, and (2) signal both axes in alternation. Participants demonstrated essentially straight-line trajectories toward the target under both instructions, but the temporal parameters (rate of approach, duration) showed an advantage for correction on both axes in sequence.

In-Situ Force Augmentation Improves Surface Contact and Force Control.

Surgeons routinely perform surgery with noisy, sub-threshold, or obscured visual and haptic feedback, either due to the necessary surgical approach, or because the systems on which they are operating are exceedingly delicate. Technological solutions incorporating haptic feedback augmentation have been proposed to address these difficulties, but the consequences for motor control have not been directly investigated and quantified. In this paper, we present two isometric force generation tasks performed with a hand-held robotic tool that provides in-situ augmentation of force sensation. An initial study indicated that magnification helps the operator maintain a desired supra-threshold target force in the absence of visual feedback. We further found that such force magnification reduces the mean and standard deviation of applied forces, and reduces the magnitude of power in the 4 to 7 Hz band corresponding to tremor. Specific benefits to stability, voluntary control, and tremor were observed in the pull direction, which has been previously identified as more dexterous compared to push.

Mental visualization of objects from cross-sectional images.

We extended the classic anorthoscopic viewing procedure to test a model of visualization of 3D structures from 2D cross-sections. Four experiments were conducted to examine key processes described in the model, localizing cross-sections within a common frame of reference and spatiotemporal integration of cross sections into a hierarchical object representation. Participants used a hand-held device to reveal a hidden object as a sequence of cross-sectional images. The process of localization was manipulated by contrasting two displays, in situ vs. ex situ, which differed in whether cross sections were presented at their source locations or displaced to a remote screen. The process of integration was manipulated by varying the structural complexity of target objects and their components. Experiments 1 and 2 demonstrated visualization of 2D and 3D line-segment objects and verified predictions about display and complexity effects. In Experiments 3 and 4, the visualized forms were familiar letters and numbers. Errors and orientation effects showed that displacing cross-sectional images to a remote display (ex situ viewing) impeded the ability to determine spatial relationships among pattern components, a failure of integration at the object level.

Real-time tomographic reflection in facilitating percutaneous access to the renal collecting system.

BACKGROUND AND PURPOSE: Real-time tomographic reflection is a novel technique that uses a geometrically fixed arrangement of a conventional ultrasound transducer, a transducer-incorporated monitor, and a half-silvered mirror. This device, dubbed the Sonic Flashlight, generates a virtual anatomically scaled image, obviating the need for a separate monitor. It may therefore facilitate invasive procedures, such as percutaneous access to the kidney. This proof-of-concept study assesses the feasibility of this technique for renal imaging and concomitant needle puncture guidance. MATERIALS AND METHODS: In a swine model with induced hydronephrosis, the Sonic Flashlight was used to visualize and guide needle access to the renal pelvis. Passage of a 7-inch, 18-gauge spinal needle was performed. Entry into the collecting system was confirmed by the aspiration of urine. RESULTS: The anechoic renal pelvis and hyperechoic needle tip could be seen with the Sonic Flashlight device. Successful access to the collecting system was obtained twice without dificulty. The sonographic image, appearing to emanate from the tip of the transducer, makes visualization and manipulation more intuitive. Furthermore, by placing the operator's eyes and hands in the same field as the sonogram, image-guided procedures are potentially easier to learn. CONCLUSION: The relatively shallow depth of penetration of the current device limits its clinical usefulness. A new Sonic Flashlight with a greater depth of penetration is in development.

Peripherally inserted central catheter placement using the Sonic Flashlight.

The Sonic Flashlight is an ultrasound (US) device that projects real-time US images into patients with use of a semireflective/transparent mirror. The present study evaluated the feasibility of use of the Sonic Flashlight for clinical peripherally inserted central catheter placements, originally with the mirror located inside a sterile cover (n = 15), then with the mirror outside (n = 11). Successful access was obtained in all cases. Results show that this new design improved visibility, as judged subjectively firsthand and in photographs. The study demonstrated the feasibility of the Sonic Flashlight and the new design to help assure sterility without degrading visibility, allowing further clinical trials involving physicians and nurses.

Fully automated common carotid artery and internal jugular vein identification and tracking using B-mode ultrasound.

We describe a fully automated ultrasound analysis system that tracks and identifies the common carotid artery (CCA) and the internal jugular vein (IJV). Our goal is to prevent inadvertent damage to the CCA when targeting the IJV for catheterization. The automated system starts by identifying and fitting ellipses to all the regions that look like major arteries or veins throughout each B-mode ultrasound image frame. The spokes ellipse algorithm described in this paper tracks these putative vessels and calculates their characteristics, which are then weighted and summed to identify the vessels. The optimum subset of characteristics and their weights were determined from a training set of 38 subjects, whose necks were scanned with a portable 10 MHz ultrasound system at 10 frames per second. Stepwise linear discriminant analysis (LDA) narrowed the characteristics to the five that best distinguish between the CCA and IJV. A paired version of Fisher's LDA was used to calculate the weights for each of the five parameters. Leave-one-out validation studies showed that the system could track and identify the CCA and IJV with 100% accuracy in this dataset.

Amygdalae morphometry in late-life depression.

OBJECTIVE: The amygdalae have been a focus of mood disorder research due to their key role in processing emotional information. It has been long known that depressed individuals demonstrate impaired functional performance while engaged in emotional tasks. The structural basis for these functional differences has been investigated via volumetric analysis with mixed findings. In this study, we examined the morphometric basis for these functional changes in late-life depression (LLD) by analyzing both the size and shape of the amygdalae with the hypothesis that shape differences may be apparent even when overall volume differences are inconsistent. METHODS: Magnetic resonance imaging data were acquired from 11 healthy, elderly individuals and 14 depressed, elderly individuals. Amygdalar size was quantified by computing total volume and amygdalar shape was quantified with a shape analysis method that we have developed. RESULTS: No significant volumetric differences were found for either amygdala. Nevertheless, localized regions of significant shape variation were detected for the left and right amygdalae. The most significant difference was contraction (LLD subjects as compared to control subjects) in a region typically associated with the basolateral nucleus, which plays a key role in emotion recognition in neurobiologic models of depression. CONCLUSIONS: In this LLD study, we have shown that, despite insignificant amygdalar volumetric findings, variations of amygdalar shape can be detected and localized. With further investigation, morphometric analysis of various brain structures may help elucidate the neurobiology associated with LLD and other mood disorders.

Vascular access: comparison of US guidance with the sonic flashlight and conventional US in phantoms.

PURPOSE: To prospectively evaluate whether ultrasonography (US)-guided vascular access can be learned and performed faster with the sonic flashlight than with conventional US and to demonstrate sonic flashlight-guided vascular access in a cadaver. MATERIALS AND METHODS: Institutional review board approval and oral and written informed consent were obtained. The sonic flashlight replaces the standard US monitor with a real-time US image that appears to float beneath the skin and is displayed where it is scanned. In studies 1 and 2, participants performed sonic flashlight-guided needle insertion tasks in vascular phantoms. In study 1, 16 participants (nine women, seven men) with no US experience performed 60 simulated vascular access trials with sonic flashlight or conventional US guidance. With analysis of variance (ANOVA) and power-curve fitting, improvement with practice rate and mean differences between techniques and tasks were examined. In study 2, 14 female nurses (mean age, 50.1 years) proficient with conventional US performed simulated vascular access trials on three tasks with the sonic flashlight and conventional US. With random assignment, half the participants used the sonic flashlight first and half used conventional US first. Mean performance with each technique and that with each task were compared by using ANOVA. In study 3, feasibility of sonic flashlight guidance for access to internal jugular and basilic veins was demonstrated in a cadaver. RESULTS: For study 1, learning rates (ie, decrease in access time over trials) did not differ for vascular access with sonic flashlight and conventional US. Overall, participants achieved faster vascular access times with sonic flashlight guidance (P < .007). In study 2, participants performed procedures faster overall with the sonic flashlight (P < .02) and found the sonic flashlight easier to use. In study 3, sonic flashlight-guided vascular access was gained in the cadaver. CONCLUSION: Learning and performance of vascular access were significantly faster with the sonic flashlight than with conventional US, and vascular access could be gained in a cadaver; the sonic flashlight is ready for clinical trials.

Optical merger of direct vision with virtual images for scaled teleoperation.

Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a half-silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to be the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

Psychophysical evaluation of in-situ ultrasound visualization.

We present a novel psychophysical method for evaluating ultrasonography based on Real-Time Tomographic Reflection (RTTR), in comparison to Conventional Ultrasound (CUS). The method measures the user's perception of the location of an ultrasound-imaged target independently from assessing the action employed to reach it. Three experiments were conducted with the Sonic Flashlight (SF), an RTTR device, and CUS. The first two experiments determined subjects' perception of target location with a triangulation-by-pointing task. Depth perception with the SF was comparable to direct vision, while CUS caused considerable underestimation of target depth. Binocular depth information in the SF was shown to significantly contribute to its superiority. The third experiment tested subjects in an ultrasound-guided needle insertion task. Because the SF provides visualization of the target at its actual location, subjects performed insertions faster and more accurately by using the SF rather than CUS. Furthermore, the trajectory analysis showed that insertions with the SF generally went directly to the target along the desired path, while CUS often led to a large deviation from the correct path consistent with the observed underestimation of target depth. These findings lend great promise to the use of RTTR-based imaging in clinical practice and provide precise means of assessing efficacy.

Intuitive intraoperative ultrasound guidance using the Sonic Flashlight: a novel ultrasound display system.

OBJECTIVE: The Sonic Flashlight (SF) is a new handheld ultrasound (US) display device being developed at our institution. It replaces the standard monitor on a conventional ultrasound (CUS) system with a miniature monitor and half-silvered mirror to reflect real-time US images into the body. With the SF, the imaged body part appears translucent, with the US image appearing to float below the surface of the anatomy, exactly where it is being scanned. The SF merges the patient, US image, instrument, and operator's hands into the same field of view, allowing the user to operate directly on the US image using normal hand-eye coordination. In contrast, CUS procedures result in displaced hand-eye coordination when the operator looks away from the patient to view the CUS monitor. Intraoperatively, the SF may make localizing and accessing tumors, foreign bodies, hematomas, vascular malformations, and ventricles easier and more accurate, especially for those without extensive CUS training. METHODS: In this cadaver study, the SF was used to visualize the brain and guide a needle into an implanted simulated tumor. The needle was inserted both in the US plane and outside of the US plane. INSTRUMENTATION: Sonic Flashlight fifth generation research prototype. CONCLUSION: The needle was easily and intuitively visualized and guided into the lesion, both within and outside of the US plane. By having the US image appear directly beneath the brain surface, the surgeon can easily and quickly guide the needle or surgical instrument to the lesion. The operator's eyes never have to leave the surgical field, as they do with CUS technology. The impact of this device on neurosurgical procedures could be significant. The ease of use, intuitive function, and small instrument size allow the surgeon to quickly localize lesions, confirm surgical positioning, and assess postoperative results.

Guidance of retrobulbar injection with real-time tomographic reflection.

OBJECTIVE: Retrobulbar and peribulbar injections are common ophthalmologic procedures used to deliver anesthetics and other medications for ophthalmic therapy and surgery. These injections, typically performed without any type of guidance, can lead to complications that are rare but visually devastating. The needle may penetrate the optic nerve, perforate the globe, or disperse toxic quantities of drugs intraocularly, causing major visual loss. Sonographic guidance may increase the accuracy of the needle placement, thereby decreasing the incidence of complications. However, difficulties arise in coordinating the relative location of the image, the needle, and the patient. Real-time tomographic reflection is a new method for in situ visualization of sonographic images, permitting direct hand-eye coordination to guide invasive instruments beneath the surface of the skin. METHOD: In this preliminary study, real-time tomographic reflection was used to visualize the eye and surrounding anatomic structures in a cadaver during a simulated retrobulbar injection. RESULT: The needle tip was easily followed as it was advanced into the retrobulbar space. CONCLUSIONS: The images presented in this preliminary study show the use of real-time tomographic reflection to visualize insertion of an invasive instrument into the human body.