COCHLEAR IMPLANT SIMULATION

version 2.0

Copyright:
Ángel de la Torre Vega
Marta Bastarrica Martí
Rafael de la Torre Vega
Manuel Sainz Quevedo
Granada, December 2004
All rights reserved.

Description and usage of the program:

1.- Hardware and software requirements

2.- Software installation

3.- What is “Cochlear Implant Simulation”?

4.- Sound perception through a cochlear implant

5.- How does “Cochlear Implant Simulation” work?

6.- How has “Cochlear Implant Simulation” been developed?

7.- What is not “Cochlear Implant Simulation”?

8.- Version history

9.- The authors


Hardware and software requirements

Minimal hardware requirements

  • Processor: Compatible Pentium.
  • RAM memory: 16 MB of RAM.
  • Disk space: 1 MB of free disk space available.
  • Screen resolution: 800 x 600 pixels.
  • Sound card: One sound card compatible with Windows, for listening to the results of simulations and recording new signals.

Recommended hardware

  • Processor: Pentium III or better.
  • RAM memory: 128 MB of RAM.
  • Disk space: 5 MB of free disk space available.
  • Screen resolution: 1024 x 768 pixels.
  • Sound card: One sound card compatible with Windows.
  • Microphone and headphones

Minimal software requirements

In order to run "Cochlear Implant Simulation" a standard PC running a 32 bits Windows operating system is necessary. In particular, it can be run under the following operating systems:

  • Microsoft Windows 95
  • Microsoft Windows 98
  • Microsoft Windows NT
  • Microsoft Windows 2000
  • Microsoft Windows XP


Software installation

"Cochlear Implant Simulation" software can be downloaded from the web page http://www.ugr.es/~atv/web_ci_SIM/en/ci_sim_en.htm. As described later, the program "Cochlear Implant Simulation" actually does not require any installation process. It is only needed to copy the executable and help files in a local folder.

In order to do that, there are two options:
  1. Download and copy the ci_sim.exe and ci_sim.chm files in a local folder, or
  2. Download the file ci_sim.zip, and decompress it in a local folder.

After copying the file/s, the application can be activated by double-clicking on the ci_sim.exe icon.


Pagina nueva 1

What is “Cochlear Implant Simulation”?

“Cochlear Implant Simulation” is a software application that allows to simulate how the sound is perceived by a subject who has received a cochlear implant. The main functions of this application are:

  • Reading a sound file in “wav” format or recording an audio signal from the computer audio input.
  • Configuring parameters which are used for simulation. Different situations with respect to sound perception by a cochlear implant user can be simulated by modifying these parameters.
  • Obtaining a new audio signal, from the original one (“wav” file or recorded sound), synthesized according to the selected simulation parameters. This new signal represents how the original sound would be perceived by the implanted subject.
  • Playing and saving original and synthesized audio signals.

The software application can be executed in a PC computer with Windows operating system. The application “Cochlear Implant Simulation” version 2.0 has been developed at the University of Granada. This version was finished in December 2004.


Percepción del sonido a través de un implante coclear

Sound perception through a cochlear implant

What is a cochlear implant?

A cochlear implant is an electronic system that is used to provide hearing to subjects affected by severe or profound hearing loss. The system consists of two main elements, an external processor and an internal element that is implanted into the patient by means of a surgical operation. The implanted element has one electrode array which is allocated into the cochlea, in order to provide stimulation of the auditory nerve by means of electrical stimuli.

The basic operation of the cochlear implant is the following: The processor has a microphone to get the sound. The processor analyzes the sound and determines the stimulation level to be sent at each electrode (and its evolution in time). The stimulation pattern is sent into the internal part of the system by a radio transmission, and the internal part generates the electrical pulses, that are presented at each intra-cochlear electrode of the implant. The pulses at each electrode cause the activation of the neural ends of the auditory nerve, and this activity is transmitted to the central nervous system, providing a hearing sensation to the patient.

Currently, cochlear implant is the only effective solution for most of the severe and profound hearing losses for which hearing aid is not enough. Today there is enough experience with this treatment (there are about 50.000 implanted patients around the world) and its efficiency is not questioned. This way, there is a considerable percentage of implanted patients who are able to communicate through telephone using a cochlear implant (that is, they are able to understand speech from the hearing provided by their implant, without any visual aid and under moderate noise levels).

How does a patient with a cochlear implant perceive sound?

Knowing with some precision how the sound is perceived with a cochlear implant is extremely difficult, because of the amount of involved factors:

  • Technological factors: technical features of the cochlear implant system (stimulation rate, disposition and number of electrodes in the implant, stimulation mode), signal processing performed from acquisition of audio to generation of stimulation, coding strategy, etc.
  • Surgical factors: allocation of the electrode array, its insertion depth, etc.
  • Physiological factors: index of surviving neurons, refractory period, dynamic range for electrical stimulation, intensity resolution, etc., and the variation of such parameters for each cochlear portion.
  • Other factors: appropriate programming map (fitted to the patient) in the processor, duration of deafness, age of beginning of deafness related to the auditory and language development, auditory experience before deafness, ability and disposition for learning and auditory training, etc.

It is well known that all these factors affect in any way to sound perception by implanted subjects, but it is difficult to evaluate the influence of each one (in relation with the others) or which one will condition the hearing perception quality with the highest influence. In order to know how a cochlear implant wearer perceives the sound we have some possibilities:

  • Testimony from patients. The information provided by patients about the quality of their sound perception is of great interest for understanding how the implant affects to hearing quality. This information has the disadvantages that it is very subjective and also that it is affected by the patient auditory experience (both, previous and following the cochlear implant activation).
  • Indirect observations. By means of tests designed for evaluating certain aspects of hearing quality of implanted patients, obtaining additional specific information is possible.
  • Analysis of audio signal transformations. By applying Signal Processing techniques we can simulate the transformations that take place in the audio signal from its acquisition to the neural ends activation. This way, the information loss associated to both, the analysis process (due to the processor, the implant and the coding strategy) and to the interaction between electrodes and neural ends, can be represented.

Purpose of “Cochlear Implant Simulation” project

The purpose of “Cochlear Implant Simulation” project has been the combination of both, the experience acquired through patients testimony and observation of their abilities, and the knowledge about technological aspects in cochlear implant systems and physiology of hearing, in order to develop an application that simulates hearing perception through the cochlear implant. This simulation system aims to verify the following features:

  • Representing the main factors conditioning the quality of hearing perception through the cochlear implant.
  • Providing a system to perform simulations with audio files in order to allow normal hearing people to hear the sound like perceived by an implanted subject.
  • The simulation system is intended to be a generic simulation platform, in order to allow simulations according to the features of the different cochlear implant systems available in the market, or even according to features of fictitious systems not available in the market.
  • The simulation system is intended to work under a user-friendly interface, allowing its usage by all the parts involved in the treatment of hearingloss with cochlear implant (the patient’s parents and relatives, teachers, speech therapists and professionals involved in rehabilitation, psychologists, technicians involved in fitting, ENT specialists, etc.).


Pagina nueva 1

How does “Cochlear Implant Simulation” work?

Using the application

“Cochlear Implant Simulation” software basically includes 4 tasks: (1) preparing the audio signal to be processed; (2) configuring the simulation parameters; (3) synthesis of processed signal; (4) saving the resulting signals. The next sections describe each of these functions.

Preparing the audio signal to be processed

The input audio signal (the signal that would be acquired by the microphone of the cochlear implant system) is prepared either opening an audio file (in “wav” format) or recording it through the sound card of the computer (for example, from the microphone).

In order to read a “wav” file the user can click on “Open” button of the application. It can also be done by selecting “Open” in the “File” menu bar or pressing the key sequence “Ctrl+O”. In all these cases, a dialog is shown, which allows navigation over the file system and identification of audio files with “wav” extension. When the file to be processed is identified, it can be read by the program by double click or by selecting the file and clicking on the “Open” button. When the audio file is read, it can be found in the “UnprocessedWAVs” list.

In order to record an audio signal, record button must be clicked (the red button in the bottom of the main window). Recording can be stopped by clicking on the grey button in the bottom of the window. The program will record from the audio input of the computer sound card and it will assign a filename to the recorded signal. The file corresponding to the recorded signal will be found in the “Unprocessed WAVs” list.

By clicking on the green button (in the bottom of the main window), the program will play the selected audio file (either recorded from sound card or read from the file system). It will be played through the audio output of the computer sound card.

Configuring the simulation parameters

The configuration of the simulation parameters is done through the “configuration dialog”, which is activated by clicking on “Configure” button. It can also be activated by selecting “Configure Parameters” in the “Preferences” menu bar or by pressing the key sequence “Ctrl+P”.

The “configuration dialog” provides access to the different parameters involved in the simulation. Parameters requiring a numerical value for their con- figuration (like rate, fMin, fMax, etc.) can be configured either by moving the corresponding scroll bar or by typing the numerical value in the corresponding cell. Parameters that need a binary option (Envelope Detection, Synchronization, etc.) are selected by clicking on the corresponding option. The meaning of each configuration parameter is described later.

The software also allows reading and writing configuration files. These files have the “.par” extension. In order to read or write configuration files the options “Read Parameters” or “Save Parameters”, respectively, must be clicked in the “Preferences” menu bar.

Performing the simulation

In order to synthesize the audio signal as it would be perceived through the cochlear implant according to the established configuration, the original file must be selected and then the “Start Simulation” button (in the bottom of the main window) must be clicked. During some time (that depends of the computer speed) the signal will be processed and then a new file will be found in the “Processed WAVs” list.

Each audio file resulting from simulation has its own set of simulation parameters used for this simulation. These configuration parameters are shown by clicking on the “+” icon beside each file.

When the simulation process ends, the new file can be selected and then a click on the play button (the green button in the bottom) plays the audio file obtained through the simulation (the synthesized signal). This way the result of the simulation can be listened through the audio output of the computer.

Storing of results

The synthesized signal has an associated file name, which is assigned automatically from the file name of the source audio file. This signal can be stored as a “wav” file by clicking on the “Save” button or by selecting “Save” or “Save As” options in the “File” menu bar. Also “Save” and “Save As” actions are activated by pressing the “Ctrl+S” and “Ctrl+Shift+S” key sequences, respectively.

If some of the audio files must be removed from the “Processed WAVs” or “Unprocessed WAVs” lists, it can be done by selecting the file to be removed and then clicking on the “Close” button (or selecting the “Close” option of the “File” menu bar). If the file was not previously saved, the program shows a warning message to prevent accidental loss of synthesized signals.

Other features of the program

Help is activated by clicking in the “Help” button, or by selecting “Contents” in the “Help” menu bar. It can also be done by pressing the “F1” key.

In order to exit the application, the user can press the key sequence “Ctrl+Q” or select “Quit” in the “File” menu bar.

Description of application commands

  • Buttons:
    • Open: It opens a dialog for selecting and reading a “wav” file from the file system.
    • Save: It saves the selected “wav” file into the file system.
    • Close: It closes the selected “wav” file, showing a warning if it was not previously saved.
    • Configure: It opens the parameters configuration dialog.
    • Help: It shows the help contents.
    • Green button (play): It plays the selected “wav” file (either processed or not processed) allowing it to be listened through the computer audio output.
    • Grey button (stop): It stops recording, when it has been previously started.
    • Red button (record): It starts recording of sound through the audio input.
    • Start Simulation: Using as source audio signal the selected one, it synthesizes the audio signal as it would be perceived by the implanted subject, according to the established simulation parameters.
  • File menu:
    • Open: It opens a dialog for selecting and reading a “wav” file from the file system.
    • Save: It saves the selected “wav” file into the file system.
    • Save As: It saves the selected “wav” file into the file system, allowing to change the file name or to save in other directory. It opens the “Save As” file dialog.
    • Close: It closes the selected “wav” file, showing a warning if it was not previously saved.
    • Quit: It closes the program.
  • Preferences menu:
    • Configure Parameters: It opens the parameters configuration dialog.
    • Read Parameters: It reads a parameters configuration file.
    • Save Parameters: It stores current simulation parameters into a file.
  • Help menu:
    • Contents: It shows the application help contents.
    • About: It shows information about the authors and the copyright.

Configuring simulation parameters

The simulation parameters (that can be modified in order to model different situations with respect to the perception of sound by the implanted patient) are listed below:

  • rate: Stimulation rate (in pulses per second) for each electrode.
  • fMin: Lower limit of the spectral range processed by the cochlear implant system, expressed in Hz.
  • fMax:Upper limit of the spectral range processed by the cochlear implant, expressed in Hz.
  • length-ci: Length of the electrode array of the cochlear implant, in mm.
  • n-channels-ci: Number of channels in the cochlear implant.
  • n-inserted-ci: Number of channels inserted in the cochlea during the surgery.
  • n-of-m: Number of electrodes which are activated in each stimulation cycle of the m electrodes switched on.
  • Interaction: Interaction coefficient (in mm) to model interaction among channels.
  • Cutoff-frequency: For the case of Electro-Acoustic Stimulation (EAS), this is the cutoff-frequency which determines the frequency range processed by the hearing aid and by the cochlear implant.
  • Envelope Detection: Procedure used for the envelope detection in the coding strategy. Two options can be selected: Hilbert+FIR (filter bank with finite impulsive response filters and envelope detection based on the Hilbert transform) or Rect-LP+IIR (filter bank with infinite impulsive response filters and envelope detection based on rectification and low-pass filtering).
  • Synchronization: The synchronization capability of the neural activity with the stimulation is modeled with this parameter.
  • Electro-Acoustic Stimulation:This parameter allows to simulate the audition of a patient for whom the electrical stimulation through the cochlear implant is combined with the acoustic stimulation through a hearing aid.
  • Frequency Shift: By switching this option on, the synthesis is performed taking into account the frequency corresponding to the position of each electrode of the implant according to the place theory.

Rate

This parameter represents the stimulation rate for each channel of the cochlear implant, that is, the number of pulses per second that are presented at each electrode of the implant. During the processor fitting, a value higher than 1000 pps (pulses per second) is recommended for this parameter, because the repolarization time of the neural ends is about 2 ms, and values lower than than 800 pps would cause an undesired synchronization of the neural activity with the stimulation pulses, instead of the desired synchronization of neural activity with the audio signal. The value assigned to this parameter strongly affects to the temporal resolution in the perception of the audio signal and significantly affects the quality of the perceived signal, particularly for those patients with a good synchronization ability. In the “Cochlear Implant Simulation” program, this parameter can be set between 10 and 10000 pulses per second.

fMin and fMax

These are the lower and upper limits of the spectral range processed by the cochlear implant system, expressed in Hz. fMin and fMax are used to build the filter bank. In the simulation (as in a cochlear implant system), the frequency components lower than fMin or upper than fMax are not processed. Taking into account the spectral range os speech, it is recommended that fMin is below 350 Hz and fMax is above 4000 Hz. In the “Cochlear Implant Simulation” program, fMin can take values between 20 Hz and 5000 Hz, while fMax can be between 500 Hz and 10000 Hz, being fMin smaller than fMax.

Length-ci and n-channels-ci

These two parameters model the geometry of the electrode array. The cochlear implant length, expressed in mm, and the number of channels of the cochlear implant, are used to determine the distance between two consecutive electrodes and the position of each electrode along the cochlea. This way, the interaction among channels (which depends on distribution of the current field around each electrode and the distance between consecutive electrodes) can be modeled. These parameters also allow to model the frequency shift associated to stimulation with cochlear implants, which will depend on the difference between the central frequency of the filter associated to each electrode and the frequency associated to the position in which this electrode is located.

In the “Cochlear Implant Simulation” program, the length of the cochlear implant can take values between 1 mm and 30 mm, while the number of channels of the cochlear implant can be set between 1 and 50.

n-inserted-ci

This parameter represents the number of electrodes inserted during the surgery. It is assumed that the cochlear implant is appropriately programmed and this will be the number of channels activated in the cochlear implant. This way, 11 in practice, this parameter is used to define the number of frequency bands in which the spectral range (defined by fMin and fMax) is divided. The number of frequency bands will condition, in principle, the tonotopic spectral resolution, and therefore the higher is the number of channels, the sound is perceived with better quality. However, the tonotopic spectral resolution is also strongly affected by the interaction among channels, because the current inserted from each electrode is not confined, and it is spread in a relatively wide region. This way, for a high number of spectral bands, the tonotopic spectral resolution is more limited by the interaction among channels than by the number of bands.

The number of inserted electrodes in the cochlear implant is also used to determine the position of each electrode along the cochlea, and this way to determine the characteristic frequency of each electrode location, in order to model the frequency shift effect. The number of inserted electrodes must be smaller than or equal to the number of channels of the cochlear implant.

n-of-m

This parameter is the number of channels activated in each stimulation cycle for n-of-m coding strategies. When the processor uses a CIS strategy, this parameter must be equal to the number of inserted electrodes (the maximum value allowed by the program).

When a n-of-m strategy is used, in each stimulation cycle (according to the stimulation rate established by the rate parameter) the energies for the different channels are compared at each cycle and only the n channels with highest energy are selected for stimulation, while the others are discarded (energy of discarded channels is set to a null value). When n is equal to m (CIS strategy) all the channels are selected in each stimulation cycle.

As the value of m is smaller, the quality of the synthesized signal is degraded, because of the information loss (due to the cancellation of the channels with lower energy). The n-of-m strategies are used to avoid an extreme reduction of the stimulation rate in cochlear implants with a high number of electrodes.

Interaction

This parameter is used to model the interaction among channels. This interaction is modeled as a transfer of energy from a given channel in the analysis block to the adjacent channels in the block of synthesis. For a given channel, “k”, a part of the current generated for the stimulation will activate the neural ends close to this electrode, but there will be also an stimulation of the neural ends close to the electrodes “k+1”, “k-1”, “k+2”, “k-2”, etc. In practice, this effect reduces the tonotopic spectral resolution. The interaction among channels becomes more important when the separation between adjacent electrodes is smaller. This way, the interaction (or current transfer) is greater between electrodes “k” and “k+1” than between electrodes “k” and “k+2”.

This parameter is used to model the interaction among channels. This interaction is modeled as a transfer of energy from a given channel in the analysis block to the adjacent channels in the block of synthesis. For a given channel, “k”, a part of the current generated for the stimulation will activate the neural ends close to this electrode, but there will be also an stimulation of the neural ends close to the electrodes “k+1”, “k-1”, “k+2”, “k-2”, etc. In practice, this effect reduces the tonotopic spectral resolution. The interaction among channels becomes more important when the separation between adjacent electrodes is smaller. This way, the interaction (or current transfer) is greater between electrodes “k” and “k+1” than between electrodes “k” and “k+2”.

Intensity(A)*exp(-Distance(A-B)/Interaction.Coefficient)

The intensity observed at the neural ends associated to electrode “B” will be the sum of the contributions from all the adjacent electrodes. When the distance between electrodes is small (compared with the interaction coefficient) there is a strong interaction among the different channels, which makes the discrimination of stimuli coming from two adjacent electrodes more difficult (and limites the tonotopic spectral resolution). Some studies provides an estimation of this interaction coefficient, which value could be around 2 or 3 mm.

Cutoff-frequency

This parameter has effect only when the Electro-Acoustic Stimulation (EAS) option is selected. Electro-Acoustic Stimulation consists on the combination of a hearing aid and a cochlear implant. The cutoff frequency defines the frequency range that is processed by the hearing aid (frequencies below the cutoff frequency) and the spectral range processed by the implant (from the cutoff frequency to the frequency fMax). In this case, the filter bank for the cochlear implant is designed using this spectral range. This parameter can take values between 20 Hz and 5000 Hz.

Envelope detection

In most cochlear implant systems, the filter bank is designed with Infinite Impulsive Response (IIR) filters and the envelope detection is performed by applying a rectifier and a low-pass filter to the output of each filter in the filter bank. In order to simulate this situation, the option Rect-LP+IIR must be selected. Other cochlear implant systems make use, for each channel, of a couple of Finite Impulsive Response (FIR) filters (which avoids phase distortion of IIR filters) and the envelope is obtained by means of the Hilbert transform (which provides a better representation of the evolution in time of the envelope). The option Hilbert+FIR must be selected to simulate this case.

Synchronization

This parameter is included to simulate the capability of synchronization of the neural activity with the presented stimulus. It can take values between 0.0 (poor synchronization) and 1.0 (good synchronization). By selecting good synchronization, a situation in which the damage of the auditory nerve is not very extensive is represented and this allows the patient to be able to extract temporal information from the stimulation pattern. By selecting bad synchronization, a situation in which the damage of the auditory nerve is more important is represented. In this situation, an important part of the temporal information is lost. Both situations are modeled through the synthesis process, by using as excitation signal for synthesis either a Gaussian white noise (in the case of bad synchronization) or a sequence of impulses located at each local maxima of the envelope for each channel (in the case of good synchronization). With this last excitation signal, the program models the fact that for a good state of the neural ends, most of the firings of the auditory nerve take place when the energy reaches a maximum value in the audio signal. Both excitation signals (Gaussian white noise and sequence of impulses) are combined according to the synchronization parameter in order to represent situations between ``poor synchronization'' and ``good synchronization''.

Electro-Acoustic Stimulation

Electro-Acoustic Stimulation (EAS) combines the electrical stimulation (through the cochlear implant) with the acoustic stimulation (through a hearing aid). This technique is useful for those patients who keeps residual hearing for low frequencies. When this parameter is selected for simulation, the audio spectral range is split into two parts: lower frequencies corresponding to acoustic stimulation (frequencies below the cutoff frequency) and upper frequencies corresponding to electrical stimulation (from the cutoff frequency to fMax). The synthesized signal in this case is the sum of the part corresponding to acoustic stimulation and the signal resulting from simulate the cochlear implant system for the spectral range defined by the cutoff frequency and fMax.

Frequency shift

The activation of this option implies that the simulation program uses a filter bank for synthesis with frequencies different than those used by the analysis. The frequencies of the filter bank used for synthesis are set according to the place theory, taking into account the allocation of each electrode along the cochlea.


How has “Cochlear Implant Simulation” been developed?

The program “Cochlear Implant Simulation” has been developed starting from a model which represents the main stages of the processes involved in sound perception by a cochlear implant patient. This model consideres both, technical and physiological aspects that will condition the sound perception. The simulation program is divided into to main blocks: one for analysis and one for synthesis.

Analysis-synthesis model

The analysis block represents the processes affecting the audio signal from acquisition from microphone to its transformation into electrical impulses generated at the different electrodes of the implant, and the generation of the action potentials by the auditory nerve.

A first part of this block just consideres the signal processing performed by the cochlear implant system, and through this part, the loss of information associated to the configuration of the cochlear implant and the coding strategy can be represented. A second part of the analysis block represents the interaction between the electrode array and the neural ends, and describes how the pattern of activity in the electrodes is transformed into a pattern of activity in the auditory nerve.

The synthesis block provides an audio signal from the pattern of activity in the auditory nerve obtained from the analysis block. The audio signal is synthesized from the pattern of activity corresponding to each frequency band (associated to each region of the cochlea). This way, the information that was lost due to the analysis process will cause a degradation in the quality of the synthesized signal. Figure 1 represents the block diagram considered for the simulation.

Figure 1: Block diagram of the program “Cochlear Implant Simulation”.

This model allows to consider the main aspects conditioning the perception through a cochlear implant, as the coding strategy, the design of the filter bank, the stimulation rate, the number of channels, the size of the cochlear implant, the allocation of the electrode array, the interaction between the implant electrodes and the neural ends, etc. The signals synthesized according to this models represent the information loss associated to the stimulation through the cochlear implant and this way this allows normal-hearing subjects to hear the sound like it would be perceived by a cochlear implant patient.

How has each effect been represented?

Signal processing in the implant

Figure 2 shows the block diagram of a conventional cochlear implant system. Audio signal is acquired by the processor microphone and then amplified. Then it is passed to a filter bank in order to separate it into different frequency bands. The output of each filter is then passed through an envelope detector. This way, for each channel, an estimation of the energy for each band is obtained as well as its evolution in time. The dynamic range adaptation block aims to transform the acoustic dynamic range for each channel into the electrical dynamic range necessary for each electrode. This transformation is specific for each patient and different for each electrode. Finally, according to the stimulation rate, the processor generates the stimulation pulses representing the current level to be presented at each electrode and at each time instant. In the case of pulsatile strategies (like CIS or n-of-m strategies) the stimulation pulses are generated in a way such that at each moment there is only one channel active, in order to avoid the problem known as “field summation”. The stimulation pattern computed by the processor is transmitted to cochlear implant and the current pulses are then generated through the electrodes of the implant.

Figure 2: Block diagram of a cochlear implant system.

The program “Cochlear Implant Simulation” processes the sound by emulating the audio signal processing performed by the cochlear implant processor, according to the set of parameters established for simulation. This provides the activity pattern at the electrodes of the cochlear implant that would be obtained when the audio signal is acquired by the microphone.

From this pattern of activity at the electrodes, and according to the model for interaction between electrodes and neural-ends, the pattern of neural activity for the groups of neural ends associated to each cochlear portion is determined. Finally, from this pattern of neural activity, the audio signal is synthesized taking into account the synchronization capability of the neural activity and the characteristic frequencies of the stimulated cochlear portions.

Stimulation rate

The stimulation rate represents the number of pulses per second generated at each electrode of the cochlear implant. This parameter limits the temporal resolution of the implant, that is, the capability for the perception of fast changes in the properties of the audio signal. As the stimulation rate is lower, the quality of the sound perceived is worse.

The temporal resolution for a cochlear implant patient is limited, in addition to the stimulation rate, by the refractory period of the neurons in the auditory nerve. The time required for the repolarization of the neurons after a neural firing is about 2 ms. For this reason, a stimulation rate above 1000 pulses per second is convenient.

In the program “Cochlear Implant Simulation”, the effect of the stimulation rate has been represented by resampling the envelopes with a sampling frequency equal to the stimulation rate. It should be considered that there are coding strategies using an updating rate lower than the stimulation rate. In that case, the value assigned to the parameter “rate” should be the updating rate and not the stimulation rate, because the updating rate represents the loss of temporal resolution.

One should consider that for extremely low stimulation rates (below 800 or 700 pps) in addition to the loss of temporal resolution there also will be an effect of synchronization of the neural activity with the stimulation pulses, which will degrade additionally the quality of perception through the cochlear implant. This effect has not been modeled in the program “Cochlear Implant Simulation”, and therefore, in the case of extremely low stimulation rates, the quality of the perception would be even worse than in the simulation.

Filter bank for analysis

The filter bank used for analysis is composed of equally-spaced filters in the logarithmic scale of frequencies in the range defined by fMin and fMax. The bandwidths of the filters are the same for all filters in this logarithmic scale of frequency, and therefore those with lower central frequencies are narrower than those with higher central frequencies.

Each channel of the cochlear implant has assigned a band-pass filter. The number of channels is specified by the parameter “n-inserted-ci”. As the number of channels is higher, the tonotopic spectral resolution is better. In the case of selecting the option “Hilbert+FIR”, the filters are designed as finite impulsive response (FIR) filters, with 100 coefficients. In the case of selecting the option “Rect-LP+IIR” 6th order Butterworth infinite impulsive response (IIR) filters are used. FIR filters present the inconvenient that their application involve more computation. On the other hand, IIR filters present the disadvantage of causing phase distortion and they could became unstable particularly in the case of narrow bandwidths.

Envelope detection

Envelope detection has been implemented either with a rectifier followed by a low-pass filter (for the option “Rect-LP+IIR”) or with the Hilbert transform (option “Hilbert+FIR”). This latter option provides an envelope which represents in an optimal way the temporal evolution of the energy in the frequency band of the filter, but has the disadvantage of requiring a couple of FIR filters in phase quadrature, with the corresponding increment in the computational load.

Strategies CIS and n-of-m

The parameter “n-of-m” allows the selection of strategies CIS (when n is equal to m, that is, equal to the number of inserted electrodes) or n-of-m strategies (when n is lower than m).

The fundamental of n-of-m strategies consists on the activation, at each stimulation cycle, of only n channels (the n channels with more energy at this cycle) of the m available channels. The objective of n-of-m strategies is to provide an increment in the stimulation rate. This is possible because the reduction of the number of channels activated at each cycle makes the duration of the whole cycle to be shorter. The increment of the stimulation rate using n-of-m has as consequence a reduction of the quality because the information corresponding to those channels with lower energy is lost.

In order to simulate the effect of n-of-m strategies, at each stimulation cycle the envelopes corresponding to the different channels are compared. The n channels with the highest energy are selected and the rest of the channels are set to a null value. This way, the information corresponding to the non selected channels is removed from the synthesized signal.

Interaction among channels

The interaction among channels has been modeled at the interface electrodes - neural ends. In a previous study the distribution of the current density field for a electrical system similar to a cochlea stimulated by a cochlear implant has been estimated. It has been found that the current inserted from an electrode is spread in a relatively wide region, for both, monopolar an bipolar stimulation modes.

When stimulation is presented at a given electrode, the ideal situation would be that only the neural ends close to this electrode were activated. However, the pulses presented at an electrode activate the neural ends close to this electrode as well as other neural ends situated at more distance. Analogously, a group of neural ends will be activated mainly by the closest electrode, but they will also be activated by other electrodes. This phenomenon could be modeled through a mixture matrix between the channels in the cochlear implant and the “channels in the auditory nerve” (being defined each of these channels as the set of neurons close to a given electrode). This way, all the channels in the cochlear implant contributes to each “channel in the auditory nerve”, and the contribution from each electrode will depend on the distance from each electrode to the considered cochlear portion. In this model we have assumed that the contribution will decay exponentially with the distance, and an interaction coefficient has been defined as the constant of this exponential decay. Some previous studies about the distribution of the current field in the cochlea suggest that an appropriate value for this constant could be around 2 or 3 mm.

In order to establish the mixture matrix describing the interaction among channels, the distance between adjacent electrodes must be considered. In order to do it, the size of the electrode array and the number of electrodes are taken into account. As the distance between electrodes is shorter (or as the interaction coefficient is higher) interaction among channels is stronger, an this cause a loss of tonotopic spectral resolution. In this case, the spectral resolution provided by the cochlear implant is not limited by the number of channels, but by the interaction among channels. As one could expect, for low values of the interaction coefficient, the quality of the synthesized signal is improved as the number of electrodes considered in the simulation is increased. However, for higher values of the interaction coefficient, the spectral resolution does not improve when the distance between adjacent electrodes is smaller than the interaction coefficient.

Synthesis of the signal

The block for synthesizing the signal correspond to the block diagram shown in figure 3. The starting point for the synthesis is the pattern of activity after including the interaction among channels. The envelope for each channel represents the energy for each time instant and for each frequency band. Therefore, in order to synthesize the audio signal, an excitation signal is taken (with an uniform distribution of energy in both, time and frequency). This excitation is filtered with a filter bank, and the output of each filter is multiplied by the corresponding envelope. The output of each channel after these operations is a signal limited in frequency (for the frequency band defining each channel) whose energy evolves in time according to the considered envelope. Finally, all the contributions (coming from the different channels) are added, and this provides an audio signal including the contributions for all the processed spectral range.

Figure 3: Block diagram of the synthesis part of the program “Cochlear Implant Simulation”.

The excitation signal to be considered can be Gaussian white noise, since this excitation presents flat spectrum and uniform distribution of energy in time. However, signals synthesized using white noise present very poor quality, because the phase of the synthesized signal is random (since the excitation used for each channel has also random phase). The result is an audio signal in which the temporal structure is lost, and particularly the fundamental tone is lost, as it cannot be perceived in the time domain. Several experiments show that most of the patients perceive the sound with better quality than that obtained by synthesizing this way. For this reason, we have proposed an alternative procedure for synthesis. This consists on using a set of impulses as excitation signal. These pulses are located at the time instants for which the envelope reaches a local maximum. An isolated impulse or a set of impulses present a flat spectrum. In order to avoid that the energy of the synthesized signal is conditioned by the excitation (it should depend on the envelopes but not on the excitation of the synthesis block) the excitation signal is normalized in order to make its energy uniform in time. Under this synthesis method, the excitation presented at each band is independent to the rest of bands and it is computed from the local maxima of the envelope of the corresponding band.

The use of an excitation like Gaussian white noise would represent how the sound is perceived by an implanted patient who, due to the damage of the auditory nerve, cannot obtain a good temporal resolution. This situation causes a loss of synchronization of the neural activity with the acoustic stimulus and the fundamental tone is not represented in the pattern of neural activity. This way of perception is present in those patients with hearing losses with more duration, or when the index of surviving neurons is lower, that is, for more extensive cochlear damage.

The use of an excitation like train of impulses would represent the perception of sound by an implanted patient who has a good capability for synchronization of the neural activity. In that case, the pattern of activity of the auditory nerve can follow the evolution of the envelope, and most of the firings takes place in the instants of time when the envelope reaches a peak of energy. This way, the fundamental tone can be perceived from the temporal pattern of activity in the auditory nerve.

In a real case, it could be expected that a patient had a perception with an intermediate quality between both situations, closer to the situation of ``poor synchronization'' when the cochlear damage is more important or closer to the situation of ``good synchronization'' when the state of auditory nerve is better preserved. In order to model this effect, the software ``Cochlear Implant Simulation'' calculates both excitation signals (Gaussian white noise and train of pulses) and combines them according to the ``Synchronization'' parameter.

Filter bank for synthesis

The filter bank used for synthesis is composed of FIR band-pass filters. FIR filters are used in order to avoid unnecessary additional phase distortion in the process of synthesis. If the option “Frequency Shift” is not activated, the central frequencies and the cutoff frequencies of the filters are the same as those in the filter bank used in the analysis block. When this option is activated, the frequencies and bandwidths of the synthesis filters are determined taking into account the allocation of each electrode and the characteristic frequency corresponding to this allocation according to the place theory. In order to do this, the size of the electrode array, the number of electrodes and the insertion depth are considered.

Electro Acoustic Stimulation

the spectrum is divided into the low part (corresponding to acoustic stimulation) and the high part (corresponding to electric stimulation). Both parts are separated taking into account the parameter “cutoff frequency”. The part corresponding to acoustic stimulation is obtained by filtering the original signal, using a low-pass filter designed for this cutoff frequency. The part corresponding to electric stimulation is obtained by processing the signal according to the configuration of the cochlear implant, where the spectral range defined for the cochlear implant is extended from the cutoff frequency to the frequency fMax. The synthesized signal is obtained by adding the part corresponding to acoustic stimulation and the part corresponding to electric stimulation.

Validation of the procedure

In order to validate the simulation procedure implemented in the software “Cochlear Implant Simulation”, some tests have been applied by presenting sentences to several patients wearing cochlear implant. These tests consisted on presenting several sentences (including synthesized and original sentences) to each patient. The patients were asked to evaluate the quality they perceived for the synthesized and original sentences.

The initial hypothesis for the validation was that both, the simulation procedure and the cochlear implant system cause a loss of quality. For the test, the implanted patient perceives the sentences after the processing performed by the simulation software and also after the processing performed by his/her own cochlear implant system (in the case of synthesized sentences) or only by the cochlear implant system (in the case of presenting the original sentence).

When the simulation is performed using a configuration providing better quality than that corresponding to the parameters of the cochlear implant system, the quality of the synthesized sentence is not affected by the simulation parameters. In this case, according to the initial hypothesis, the patient should perceive the synthesized sentence with the same quality as the original sentence. When the simulation is performed with a configuration providing worse quality than the parameters of the cochlear implant system, the quality of the synthesized signal is conditioned by the simulation parameters. In this case, the patient should perceive the synthesized sentence with worse quality than the original one.

This way, if all the simulation parameters are set to those in the cochlear implant system for a given patient, except one of the parameters, which is modified from a “good” value (providing better quality) to a “bad” value (providing worse quality) and the quality is represented in a plot versus this parameter, a curve will be observed, where for good values of the parameter the quality tends to be good (similar to that for the original sentence) and for bad values there is a fast degradation of the quality (the synthesized signal is clearly perceived worse than the original). This curve should present a knee for the value when the simulation parameter coincides with the value of the parameter in the cochlear implant system. If this is verified, we can conclude that the simulation models appropriately the effect of this parameter over the hearing quality in a cochlear implant patient.

Validation method

Validation tests have been passed to 7 patients wearing a cochlear implant. All of them were implanted at the ENT service of Hospital La Paz, Madrid, with a MED-EL Combi40+ device. The validation tests were focused on 3 simulation parameters: the stimulation rate, the number of channels and the inter-channel interaction coefficient. For each parameter, several sentences were synthesized with different values for the parameter to be studied and both, original and synthesized sentences were presented to the patient, who were asked to evaluate the quality of perception of each sentence in a scale from 0 (worst quality) to 10 (best quality).

For the analysis of results, the score for the quality of each synthesized sentence has been normalized by dividing it by the score assigned to the corresponding original sentence. This way, if a synthesized sentence presents a normalized score of 1, it must be interpreted that the patient perceives the synthesized sentence with the same quality as the original one. For each studied parameter, the normalized score has been represented versus the considered parameter. A polynomial fitting (with order 3 and with a minimum squares criterion) has been performed to these data in order to obtain a function fitting the data as well as the corresponding 95% confidence interval.

Results

Figure 4 shows the quality normalized scores versus the stimulation rate considered for simulation. Each point in the plot represents the evaluation of a synthesized sentence by a patient. A fitting of these data is also shown (minimum squares polynomial fitting) as well as the corresponding 95% confidence interval. It can be observed that for high stimulation rates, patients do not perceive a degradation in the quality of the synthesized sentence, and when the stimulation rate is reduced the quality is smaller, and very low quality is observed for rates below 700 pps. There is a knee effect in the plot corresponding to each patient, being the rate of the knee different for each patient (according to the stimulation rate programmed in his/her processor). This result validates the simulation procedure with respect to the stimulation rate.

Figure 4: Fitting of the normalized quality score versus the stimulation rate.

In order to verify the influence of the stimulation rate in the simulation related to that programmed in the patient processor, a new fitting of the data has been performed using the normalized stimulation rate as independent variable, i.e., the stimulation rate used for simulation divided by the stimulation rate programmed in the processor. The results of this fitting are shown in figure 5. In this case the knee effect is observed for a normalized stimulation rate close to one, i.e., when the stimulation rate for simulation approximates the stimulation rate programmed in the processor for each patient.

Figure 5: Fitting of the quality normalized score versus the normalized stimulation rate.

Figure 6 shows the fitting between the normalized quality score and the number of channels used for simulation. The patients had programming maps in the processor with a number of activated electrodes between 9 and 12 (2 patients with 9 electrodes, 1 with 10, 1 with 11 and 3 with 12).

In these plots, the knee effect is also observed, being the quality of the synthesized sentence similar to that of the original sentence for a high number of channels in the simulation and a fast degradation is produced in the quality when the number of channels in the simulation is smaller than 8. It is interesting to remark the fact that the knee is not around the number of channels for each patient, but always around 8 channels. This shows that the tonotopic spectral resolution obtained by the patients is not conditioned by the number of electrodes but by some other phenomenon. The tonotopic spectral resolution in the perception of sound is equivalent to having around 8 channels, in spite of having more channels. The reason of this limitation in the tonotopic spectral resolution is probably the interaction among channels.

Figure 6: Fitting of the normalized quality score versus the number of channels.

In order to evaluate the effect of the interaction among channels we have prepared tests in which the channel interaction coefficient has been modified for the simulation. The results are shown in figure 7. It can be observed that when the signal is synthesized with a small interaction coefficient, the quality of the synthesized sentence is perceived similar to that of the original sentence, but as this coefficient is increased, the quality of the signal is significantly degraded. The knee in these plots is observed for a coefficient around 1 or 2 mm, which suggests that the interaction between electrodes and neural ends can be modeled by means of the channel interaction coefficient by assigning a value close to 1 or 2 mm to this coefficient. This value is consistent with some previous observations and theoretical studies about the distribution of the density of current in the electrical system corresponding to the cochlea and the cochlear implant.

Figure 7: Fitting of the normalized quality score versus the channel interaction coefficient.

Acknowledgements

The authors acknowledge the collaboration provided by the ENT team of Hospital La Paz (Madrid, Spain), as well as that provided by the patients who participated in the validation tests.


Pagina nueva 1

What is not “Cochlear Implant Simulation”?

The aim of the software “Cochlear Implant Simulation” is to simulate the auditory perception through the cochlear implant. This simulation is based on a series of considerations and models. We could affirm that the signals synthesized by this software represent appropriately the perception through the cochlear implant if the considered models represent appropriately the process of perception and the selected parameters are set to the appropriate values, according to the situation to be modeled. The results of the validation tests guarantee the adequation of the synthesized signals as an approximation to the perception through a cochlear implant. However, it is almost impossible to establish in an unequivocal and definitive way that a synthesized signal represents exactly the perception through the cochlear implant. In that sense, the results of this simulation should be considered with some prudence.

The aspects not considered in the simulation that could influence the perception make that the synthesized signals do not represent exactly the perception through the cochlear implant. Among others aspects not modeled in the developed software, the next ones can be enumerated:

  • The fitting map in the processor. In the simulation, it is assumed that the fitting map is perfectly adapted to the requirements of the implanted patient, i.e., there is not loss of information due to an improper adjustment of the processor.
  • The resolution in intensity. In the simulation software, the limitation of the resolution in intensity associated to the neural ends is not modeled, and therefore the degradation in the quality associated to a limited resolution in intensity is not represented. This would be particularly important in the case of patients with a low index of surviving neurons in the cochlear nerve.
  • Variation of the physiological parameters along the cochlea. An adequate and accurate model of the interaction electrode - auditory nerve should consider that the channel interaction coefficient could be different for the different cochlear partitions. It also should consider that the refractory period of the neurons, the percentage of surviving neurons, the resolution in intensity and other parameters could present variations along the cochlea.
  • Modeling abnormal cochleas, like ossified or malformed cochleas. In the simulation software, we have assumed that the implant is allocated into a permeable cochlea and that the electrodes are allocated being the first one of the electrode carrier in the most apical position, and the last one in the most basal position, which allows the estimation of the characteristic frequency associated to each electrode according to the place theory. The allocation of the electrodes in the case of ossified cochleas (when the trajectory of the implant does not follow the scala tympani or when the a cochlear implant with a split electrode carrier is used) or in the case of malformed cochleas (like common cavities) is difficult to be modeled and has not been considered in the developed software.

Other important aspect to be considered is the fact that in this simulation we have estimated what would be the pattern of activity in the neural ends of the auditory nerve, and this pattern has been considered the starting point for the synthesis of the audio signal. However, in this software we did not make any consideration about the propagation of the neural activity through the auditory pathway or what is the signal processing affecting the pattern of action potentials when they are transmitted through the auditory brainstem. Modeling this aspect would be extremely complex due to the limited knowledge about the role of the auditory brainstem in hearing perception. It could be assumed that in the case of patients with hearing experience, the signal processing performed in the auditory brainstem would be “normal” (similar to that for normal-hearing subjects) but in the case of absence of hearing experience, the lack of maturation of the auditory pathway would cause an additional loss of information, not considered in the simulation software. The development of the hearing skill by the implanted patients (or the signal processing, or processing of information at the cortical level) would be also difficult to be modeled (and has not been modeled) and also affects the way in which the sound is perceived through the cochlear implant.

All these aspects must be taken into account when one affirms that the synthesized signals represents, in some way, the perception of sound through the cochlear implant. In future revisions of the software “Cochlear Implant Simulation” we plan to include some of the above described effects and factors.


Version history

Version 2.0

  • Improvements
    • The parameter "Synchronization" has been modified in order to allow continuous values between 0.0 (poor synchronization) and 1.0 (good synchronization).
    • The format of configuration files has been modified in order to support the previous improvement. The old configuration files can be read with version 2.0 (backward compatibility).
    • The documentation has been revised and updated.
  • Corrections
    • A bug in the internal sample rate conversion of the audio files has been fixed. In some cases it could cause unexpected frequency shift in the synthesized signal.
    • A bug related to the memory management has been fixed. The new version optimizes memory usage.

Version 1.0

First published version of the program "Cochlear Implant Simulation", finished the 21st June 2004.


Los autores

The authors

The software “Cochlear Implant Simulation” version 2.0 has been developed by a multi-disciplinar team integrated by:

  • Ángel de la Torre Vega:Departamento de Electrónica y Tecnología de Computadores, University of Granada, Spain.
  • Marta Bastarrica Martí: Departamento de Electrónica y Tecnología de Computadores, University of Granada, Spain.
  • Rafael de la Torre Vega: Escuela Técnica Superior de Ingeniería Informática, University of Granada, Spain.
  • Manuel Sainz Quevedo: Departamento de Cirugía y sus Especialidades, University of Granada, Spain. Head of the ENT Service of Hospital Universitario S. Cecilio, Granada, Spain.

For the development of the graphical interface, the authors have used the library wxWidgets version 2.4.2 (http://www.wxwidgets.org).

For programming reading and writing wav files, the authors have used the files wave.cpp, wave.h, rifffile.cpp and rifffile.h, copyrighted by Timothy J. Weber (http://www.lightlink.com/tjweber).

The version 1.0 has been concluded in the University of Granada, Spain, on December 2004.

Copyright:
Ángel de la Torre Vega
Marta Bastarrica Martí
Rafael de la Torre Vega
Manuel Sainz Quevedo
Granada, December 2004
All rights reserved.

The authors hereby allow the distribution, copy and usage of this software with no profit motive in mind. When this software is copied or distributed, both the executable file and the help file must be included. The modification of the executable or help files, the distribution of modified versions or the application of methods for inverse engineering to this software is not allowed without explicit permission from the owner of the copyright.

The authors distribute this software without any type of guarantee since the distribution is free. The authors are not responsible of any damage derived from its usage.