Touchscreens have been studied and developed for a long time to provide user-friendly and intuitive interfaces on displays. This paper describes the touchscreen technologies in four categories of resistive, capacitive, acoustic wave, and optical methods. Then, it addresses the main studies of SNR improvement and stylus support on the capacitive touchscreens that have been widely adopted in most consumer electronics such as smartphones, tablet PCs, and notebook PCs. In addition, the machine learning approaches for capacitive touchscreens are explained in four applications of user identification/authentication, gesture detection, accuracy improvement, and input discrimination.
This paper is organized as follows. Section 2 addresses the overview of the touchscreen technologies, and then Section 3 describes various studies on capacitive touchscreen applications that are integrated in most smartphone and notebook displays. Section 4 shows the ML approaches working with existing capacitive touchscreen technologies. Section 5 concludes this paper with some suggestions of the future directions.
There have been brief reviews of touchscreen technologies [ 76 , 96 ]. Walker [ 165 ] published many overview papers about a variety of touchscreen technologies from resistive to optical and electromagnetic resonance (EMR) stylus schemes. Those papers explained their histories, principles of operation, pros and cons, and applications. However, the technological details have not been handled such as algorithms, driving circuits, and ML approaches. Kwon et al. [ 166 ] reviewed capacitive touchscreen technologies including sensors, driving circuits, sensing methods, and stylus schemes in more detail. However, ML approaches were not introduced. Bello et al. [ 164 ] summarized ML approaches to improve security on touchscreen devices without addressing the touchscreen technologies. A variety of ML applications only for the security issues were addressed. This paper provides a unified and broader view of the touchscreen technologies with the detailed explanation and ML approaches in various scenarios.
With competitive price and timely delivery, Touch-China sincerely hope to be your supplier and partner.
There have been also efforts to integrate machine learning (ML) approaches into touchscreen technologies. These ML networks are employed to add extra input tools, to improve the touch-sensing performance, to support the user identification/authentication, to discriminate finger-touches from others, and to capture the gestures [ 135 , 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 , 144 , 145 , 146 , 147 , 148 , 149 , 150 , 151 , 152 , 153 , 154 , 155 , 156 , 157 , 158 , 159 , 160 , 161 , 162 , 163 , 164 ].
On top of the role of a visual information provider, displays have supported the interaction with users by means of various user interfaces. Users can adjust the visual information on the screen by themselves. The very old but still popular representative user interfaces are mouse and keyboard [ 64 , 65 , 66 ]. There have also existed pen tablets for more elaborate works such as drawing and writing [ 67 , 68 , 69 , 70 ]. Because these devices work on the different planes separated from displays, additional markers such as cursors and pointers are needed. On the other hand, more intuitive input interfaces called touchscreens have been studied to directly interact with displays by touching displays [ 71 , 72 , 73 , 74 ]. Touchscreen technologies can be categorized into finger-touch and stylus-touch methods. While finger-touch methods include resistive, capacitive, acoustic wave, and optical approaches [ 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 ], stylus-touch ones cover up to electromagnetic resonance (EMR) schemes including finger-touch methodologies [ 102 , 103 , 104 , 105 , 106 , 107 , 108 ]. Recently, as wearable devices such as smartwatches and smartbands are becoming more popular, small-size displays are becoming further widespread with touch sensing functionality. However, because this very small-area screen cannot support multiple finger-touches and the whole area is covered even by a single finger, a variety of separate input modalities in the outside of the screen have been studied by using infrared (IR) line sensors, microphones, gaze trackers, IR proximity sensors, electric field sensors, deformation sensors, magnetic field sensors, and mechanical interfaces [ 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 ]. In addition, some approaches have coped with the limitation of the single touch by differentiating palm and finger or identifying pad, nail, tip, and knuckle of a finger [ 124 , 125 ]. Especially, because AR/VR displays are placed near to eyes, it is impossible to touch the screen directly. Therefore, other input tools using various sensors such as leap motion sensors, electromyograph sensors, inertial measurement units, eye-trackers, IR facial gesture sensors, cameras, and axis-tilt sensors, have been employed [ 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 ].
Human beings collect a lot of information through their eyes, and many displays around us play a key role to transfer this visual information. Displays have evolved dramatically from cathode-ray tube (CRT) [ 1 , 2 , 3 , 4 ] via plasma display panel (PDP) [ 5 , 6 , 7 , 8 , 9 , 10 ] and liquid crystal display (LCD) [ 11 , 12 , 13 , 14 , 15 ] to cutting-edge organic light-emitting diode (OLED) [ 16 , 17 , 18 , 19 , 20 , 21 , 22 ] and micro-LED technologies [ 23 , 24 , 25 , 26 , 27 , 28 ]. This evolution has led to larger screen-size, slimmer design, lower weight, higher resolution, faster frame rate, brighter luminance, wider color gamut, longer life time, and lower power consumption in the large-size display applications such as monitors, televisions (TVs), and digital signage [ 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ]. The resolutions of off-the-shelf displays have increased up to 8K (7680 × 4320) along with the high frame rate of 120 Hz and the larger screen sizes than 55-inch have taken more than 30% of overall TV set sales [ 40 , 41 ]. Even rollable OLED TVs were demonstrated in the consumer electronics show 2018 (CES2018) [ 42 ]. On the other side of the small-size display applications, higher density of pixels, narrower bezel, flexibility, bendability, rollability, and low power consumption have been achieved along with enhanced picture quality [ 43 , 44 , 45 , 46 , 47 , 48 ]. The latest smartphones contain the bezel-less screens of larger pixel densities than 450 pixel per inch (ppi) and smartphones with foldable displays are being sold on the market [ 49 ]. Recently, as augmented reality and virtual reality (AR/VR) attract substantial interest, the demand for high-performance near-eye displays is increasing further [ 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 ]. Consequently, the very high resolution OLED on silicon (OLEDoS) displays up to 4410 ppi have been reported [ 58 , 59 , 60 , 61 , 62 , 63 ].
The other IR-based schemes such as planar scatter detection (PSD) [ 100 , 101 ] and frustrated total internal reflection (FTIR) [ 75 , 95 , 98 , 99 ] are similar to the acoustic wave approaches except for the use of IR instead of the sound wave. In the PSD, while the transmitters send the IR lights through the wave guide at the total internal reflection (TIR) condition, receivers sense them. When any touches are applied on the wave guide plate, it breaks the TIR condition out, therefore, the scattered and remaining TIR lights arrive at multiple receivers as described in a, leading to the extraction of the touch location by the complex analysis. The PSD can support multi-touch and high image clarity, but the larger-size touchscreens require higher computational power to extract the touch location. The FTIR also makes use of the TIR condition, but the touch location is attained from the lights escaped toward the opposite plane to the touched one as depicted in b. Those lights are captured by the external camera or vision sensors and the resultant images provide the information of touch locations. There also exist the embedded LCD solutions, where IR transmitters are allocated in the backlight and the vision sensors are placed in the pixel areas.
The optical touchscreens are developed based on the invisible infrared (IR). The traditional IR-based touchscreen places transmitters at two sides and receivers at their opposite sides without any additional layers. Because the touches block the light path over the screen between a pair of transmitter and receiver, x-axis and y-axis coordinates can be obtained by finding the receivers’ positions that do not receive IR. While large-size displays and excellent optical clarity can be supported, the bezel needs some height over the screen for IR transmitters and receivers and the multiple touches cause the ghost touch issue.
To cope with the requirement of the prior process to store the bending wave data in the APR, the DST extracts touch locations directly only from the measured bending wave data. Because the signal delay is affected by its frequency, the measured time and frequency information is used to reconstruct the bending wave pattern on the screen, which is converted to the touch coordinates. However, it also has several drawbacks such as only single touch support, high tapping strength, measurement variance, mounting dependency, as well as high computational power. In addition, both APR and DST cannot support the holding function because only the tapping action generates the sound waves.
The another one is a bending wave scheme, where the sound wave caused by tapping on the screen is used as the sound source as well as the touch signal [ 88 , 89 , 90 ]. There are two methods of acoustic pulse recognition (APR) and dispersive signal technology (DST) [ 180 ]. The APR senses the bending waves by multiple piezoelectric transducers and processes them with the data stored in the memory to extract the touch positions. Therefore, the APR needs a prior process to sample and store the large amount of bending wave data at enough number of positions over the screen. However, because the bending wave characteristics are not deterministic, the resultant coordinates have some variance, leading to errors on the location estimation. Furthermore, the enough bending wave strength is required for the sensors to detect. The bending wave characteristics are dependent of the mounting structure and material. Since too large an amount of data is necessary for multi-touch cases, it supports only a single-touch input.
The acoustic wave scheme is composed of a wave guide, sound wave sources, and receivers. The well-known technology is a surface acoustic wave (SAW) touchscreen as depicted in [ 85 , 86 , 87 , 91 ]. The SAW contains two pairs of ultrasonic transmitters and receivers to calculate x-axis and y-axis coordinates of touch locations, respectively. The reflectors in the bezel area generate multiple horizontal and vertical acoustic wave paths that have different arrival times at receivers. When a finger is placed in a certain path, the signals attenuated by that touch arrive at the receiver with corresponding delays that are converted into the position coordinates. Because the SAW needs only one wave guide layer, it has the most excellent optical performance. In addition, large size touchscreen and high durability are achievable. However, its sequential estimation of x-axis and y-axis coordinates gives rise to the same ghost touches as the self-capacitance method. It can also detect some input tools of soft materials to absorb waves and the sensing performance is sensitive to contaminants on the screen.
While additional touchscreen panels on the displays require further electronics, the embedded touchscreen solutions that are called an in-cell touch can merge panel and touchscreen electronics into a single driver integrated circuit. Therefore, various in-cell approaches have been developed including self-capacitance cells and capacitive sensors embedded in pixel areas [ 174 , 175 , 176 , 177 , 178 , 179 ].
On the other hand, because the mutual capacitance measures the overlap capacitance separately between vertical and horizontal conductive patterns, it can support multi-touch functions without any limits on the number of fingers. Therefore, it has become the widely used touchscreen technology today. The excitation pulses are applied to horizontal patterns and the transferred charges are measured through charge amplifiers at the ends of the vertical patterns. Since the amount of transferred charges is proportional to the mutual capacitance, the variation of capacitance can be detected. Section 3 will address the mutual capacitance approaches in more details.
The self-capacitance estimates x-axis and y-axis coordinates sequentially by measuring the capacitance of vertical and horizontal electrodes over the ground, respectively. Consequently, the multiple touches may cause ghost touches. For example, when there are two touches at locations of (x1, y1) and (x2, y2), the self-capacitance can figure out that there are touches at x1, x2, y1, and y2, separately, and then it provides two correct locations of (x1, y1) and (x2, y2) along with two additional ghost locations of (x1, y2) and (x2, y1) by four possible combinations of two x-axis data and two y-axis data. Thus, the self-capacitance has difficulty to support multi-touch functionality. To cope with this ghost touch issue, some panel makers use separate self-capacitance cells directly connected to the touchscreen controller that senses each capacitance variation, respectively, [ 174 ]. This approach has been implemented in the off-the-shelf smartphones.
In general, the projected-capacitive touchscreen panels use two patterned conductive layers that are separated and crossed to each other in the shape of a matrix. Horizontal and vertical patterns correspond to the position information of the touch event. While the self-capacitance senses the capacitance between layers and ground as shown in a, the mutual capacitance measures the capacitance at the overlapped areas of horizontal and vertical patterns as presented in b. Consequently, the finger touch increases the self-capacitance due to the additional parasitic capacitor in parallel and decreases the mutual capacitance due to the electric field loss by the finger placed between two electrodes.
The projected-capacitive methods can be further divided into self-capacitance and mutual capacitance architectures. Especially, the mutual capacitance has been the mainstream technology used in most consumer electronics such as smartphones, tablet PCs, and notebook PCs since the appearance of iPhones in 2007, because it can support multi-touch functions along with high durability and good optical clarity.
The capacitive scheme is divided into surface-capacitive [ 81 , 83 ] and projected-capacitive methods [ 82 , 84 ]. Surface-capacitive touchscreens consist of one conductive layer of which four corners are connected to four perfectly synchronized alternative current (AC) voltage signals as described in . While any difference does not occur without touches at these voltage sources, the finger touching the screen brings out the current difference in four voltage sources. As the voltage source is located nearer to the touch point, the current variation becomes larger due to the smaller resistive load. As a result, the touch locations are extracted from the ratio of the currents over four voltage sources. Even though it cannot deal with multiple touches at the same time, its high durability enables the integration in automated teller machines (ATMs).
Capacitive touchscreens sense the change of the capacitance caused by the finger to estimate the touch position. While resistive schemes need the pressing force to make the actual contact between two conductive layers, capacitive methods can obtain the capacitance change just by the light touch on the screen. Consequently, it enables the smooth and fast scrolling, high durability, and excellent optical performance. In addition, any materials can be adopted for layers, for example, glasses and plastics, while resistive technologies require one flexible layer at least. Because the parasitic capacitance added by fingers is very small, large-size capacitive touchscreen panels are very difficult to implement and contaminants such as water and dusts can be also recognized as touches. Recently, the large size capacitive touchscreens have been reported based on the metal mesh structure [ 108 , 173 ]. It can support only capacitive input tools including fingers to make parasitic capacitors with electrodes of the touchscreen panel.
On the other hand, there have been efforts to support multi-touch capability. Some researchers were trying to add the multi-touch functionality to a conventional structure by sensing the current consumption at voltage sources [ 167 , 168 , 169 ]. Whereas, other researchers divide the conductive films into multiple lines and columns that give rise to many separate overlapped areas [ 170 , 171 , 172 ], where each area can detect touches separately. This scheme is named as the digital resistive touchscreen [ 165 ]. Since the resistive touchscreen methods fall short of the capacitive schemes, the resistive touchscreen panels are being applied to the limited areas such as toys, office electronics, and card payment machines.
The advantages of the resistive touchscreen technology are to be able to work with anything, to be fabricated at the lowest cost, to be insensitive to any contaminants, and to consume low power. However, it has drawbacks of the only single touch support, the poor durability due to scratches, poking, and sharp objects, the poor optical clarity, and the relatively high touch force requirement [ 80 , 165 ].
An analog resistive scheme is the oldest touchscreen technology [ 165 ]. It extracts touch coordinates by sampling the voltage at the touched area. The voltage is proportional to the location of the screen due to the voltage division based on the ratio of resistances from the current position to two opposite sides [ 78 ]. The most popular resistive touchscreen panels are fabricated by 4-wire and 5-wire architectures [ 79 ]. Both methods estimate x-axis and y-axis coordinates of a touch position sequentially. Normally, two separate layers are coated by the conductive films only at one side, and one layer should be composed of a flexible material. When the touch force is applied, the flexible layer is pressed to contact the other layer and to obtain the voltage at the contacted area. Four-wire structures use both layers to generate the voltage slopes as well as to sense the voltage as illustrated in a. For example, after the flexible layer (Layer #1) generates the voltage slope at an x-axis and the other (Layer #2) senses the voltage, Layer #2 generates the voltage slope at an y-axis and Layer #1 senses the voltage. Five-wire ones apply voltages only to one specific layer (Layer #2) and use the other layer (Layer #1) only to sense the voltage as depicted in b. Therefore, it is known that 5-wire schemes usually have a longer life time.
In this section, touchscreen technologies for finger as well as stylus have been simply addressed in terms of principles of operation, advantages, and drawbacks. We categorize the touchscreen technologies into four categories of resistive, capacitive, acoustic wave, and optical, and address further various techniques in each category as shown in . compares their specifications.
3. Main Research Trends in Mutual Capacitance Capacitive Touchscreen Technologies
As explained in the previous section, there have exist various touchscreen technologies by means of resistance, capacitance, sound wave, and IR. Among them, the capacitive touchscreen has become a mainstream scheme, especially, the mutual capacitance touchscreen is the most widely used technology on many consumer electronics such as smartphones, notebook PCs, tablet PCs, and smartwatches, because of its multi-touch support, slim form factor, high optical quality, excellent durability, smooth scrolling, and so on. Particularly, this section addresses the mutual capacitance capacitive touchscreens in more details. Unlike the self-capacitance method where the parasitic capacitor of a finger touch is connected to the self-capacitor in parallel, the mutual capacitance scheme experiences the capacitance reduced by electric field leakages into a finger. As a result, the touch location can be found out by searching the position which mutual capacitance is reduced.
As shown in , a conventional mutual capacitance capacitive touchscreen panel is composed of excitation (EX) electrodes and sensing (SE) electrodes, which give rise to the mutual capacitor array at their intersection areas [181,182]. Excitation drivers generate EX pulses sequentially in the way of line-by-line that arrive at charge amplifiers attached to SE lines through mutual capacitors. The non-inverting input terminals of these charge amplifiers are connected to the reference voltage (VREF) and the charge transferred through a mutual capacitor (Cm) is converted through a feedback capacitor (Cf) into analog voltages (VOUT) that are proportional to the mutual capacitance as presented in Equation (1). VEX is the amplitude of the EX pulse. When a user touches on the screen with a finger, the reduction on the mutual capacitance is sensed as the different output voltage of the charge amplifier from the voltage level obtained without any touches as illustrated in . To improve the precision of the touch detection, the transferred charge is accumulated at the charge amplifiers over multiple EX pulses. In addition, a multiplexer (MUX) allows one analog-to-digital converter (ADC) to sample the output voltages of charge amplifiers in all SE lines sequentially. Finally, a host processor handles the digital data to determine the touch locations and it also controls excitation drivers.
VOUT=VREF−CmCfVEX.
(1)
The mainstream studies in mutual capacitance schemes are (a) improving signal-to-noise ratio (SNR) to achieve higher accuracy as well as robustness over the noises and (b) utilizing additional input tools such as styli besides fingers. In addition, it is another research trend to integrate the pressure-sensing capability. However, the most approaches support this pressure sensing function through additional sensors [183,184], the separation distance changes [185,186], or the internal circuit of the stylus [108,187]. Because additional sensors and separation distance changes are out of this review’s scope, the stylus technologies are addressed along with their pressure sensing schemes.
If you are looking for more details, kindly visit Five-Wire Resistive Touch Screen for Interactive Teaching Systems.
Comments
Please Join Us to post.
0