Augmented and Virtual Reality at Display Week: Game On!

In recent years, virtual reality has moved from science fiction movies, to academic research labs, to product development in the industry, and finally into the hands of consumers in the real world. A number of marquee devices have been launched in the market along with some compelling immersive applications. At the same time, some cool augmented reality devices and developer kits have been released as well. The pace of progress in both virtual and augmented reality technologies has been rapid.

So, in line with this fast-emerging trend in the ecosystem, SID decided to create a special track on Augmented and Virtual Reality for Display Week 2016. The rich lineup included a short course, a seminar, a number of invited and contributed presentations in the symposium, and demonstrations on the exhibit floor.

It is just what the display industry needed to be on the verge of a massive rejuvenation!

Displays are the face of some of the most used electronic devices in our daily lives – such as the smartphone, tablet, laptop, monitor, and TV, among numerous other examples. As such, the health of the display industry rises and falls with the growth and saturation of these devices. Take the exciting phase of innovation in LCD TV technology as an example. The screen size went from 24 in. to 32 in. to 40 in. to 55 in. to 80 in. and above! The resolution went from 720p to full HD to QHD and beyond, whereas the frame rates went from 60 to 120 frames per second. And there were many more advances – contrast, brightness, color, etc. However, it gets to a point where further advances in display technology provide only small incremental benefits to the consumer. This often leads to a reduced demand for new features and a slowdown in the development.

Let’s now turn to virtual reality. It’s a completely different story at the moment. The displays on the best, state-of-the-art, VR devices today fall way short of the specifications required for truly immersive and responsive experiences, despite the dizzying pace of development. The pixel density needs to increase significantly and latencies must be reduced drastically, along with many other improvements such as increased field of view, reduced pixel persistence, higher frame-rates, etc. Besides the display, the systems also require integration of accurate sensing and tracking technologies. Augmented reality devices impose additional requirements.

So this is exciting for the researchers and engineers in the industry. Back to solving some difficult challenges, with the potential for big returns. Judging by the excellent quality of the papers, presentations, and exhibitions at the Display Week, it’s obvious the display ecosystem is all geared up. Game on! – Achin Bhowmik

Game Changer: CLEARink Shows Video-Rate Reflective Display

There are good very-low-power monochrome reflective displays with slow redraw times and, with the introduction of E Ink's color display, there is now a good low-power color reflective display with very slow redraw times.

What we have not had is a reflective video-rate display, and for good reasons.  The only reflective technology that has proved to have both broad application and business feasibility has been electrophoretic  (think E Ink), and electrophoretic displays operate by moving charged particles slowly through a significant fluid layer.  The redraw time cannot be fast.  (Well, it can be faster, but then the charged particles collide violently and tear each apart, with unfortunate results.)

CLEARink has turned the conventional electrophoretic model on its head.  Very very briefly, the CLEARink display has a thin optical plate with lenslets on the inner surface.  In the white state, incoming light experiences total internal reflection (TIR) and returns to the viewer.  Reflectivity is an impressive 60%.

How does the display form a black pixel?  Lurking behind the optical plate in an "ink" are black particles that are moved toward or away from the plate.  When the particles touch the plate (that's a bit sloppy, but close enough for a blog), the TIR is defeated and light at that point is absorbed.

Clever, you say, but it's still electrophoresis, with a particle being moved through a fluid.  How can that produce video rate?  Because there's something I haven't mentioned yet.  The particle only has to move through 0. 5 micron to be "touching" or "not touching" the plate, and that very small distance can be traversed rapidly.

All of this has been public for a least several weeks, but at Display Week, the company showed technology demonstrations in its suite.  To demonstrate the monocrome video-rate display, CLEARink engineers had purchased a Kobo eReader, and simply replaced the E Ink imaging film with their own.  With the application of a video signal, the display showed very clean, 30 fps video with the subjectively good contrast and that bright 60% reflectivity.  CEO Frank Christiaens to the opportunity to note that the technology is compatible with pretty much any backplane and requires no precision alignment.

Although my colleague Bob Raikes and I were extremely impressed with this demo, Christiaens didn't want us to neglect that fact that color via matrix color filter is part of the company's mid-term road-map.  Demos were effective.  Using an MCF with an otherwise monochrome EPD has not been a satisfying approach in the past because too much of the reflective light was absorbed.  The difference here is that CLEARink starts out with 60% reflectivity rather than 40%.

So, said Christiaens, CLEARink will soon be providing something that has never before been available:  a reflective color video-rate display.

Walking back to the Moscone Center after our meeting, Raikes and I agreed that the term "game-changing" is used far too often, but that it legitimately applies here.  This fast EPD can enable new applications that cannot be realized by existing display technologies. -- Ken Werner

Wearable Displays Sport Classic Designs

There was a time when watches seemed to go out of fashion. Everyone knew the time by looking at their mobile phone screen. In the last couple of years, “connected watches” have become a wearable part of the mobile ecosystem, as their design has approached that of classic wristwatches. The intuitive round-faced hand dial watch user interface has pulled through, once again.

JDI Memory-in-Pixel reflective connected watch display. (Photos by Jyrki Kimmel.)

How has this development come about? Weren’t we satisfied with the function of the square-screen Android devices that appeared on the market about 5 years ago? Apparently not.

The wearables offering on the exhibition show floor featured many round-faced watch-sized displays. The Withings activity monitor, for instance, was featured in the E Ink booth. It sported a reflective e-paper display, in a round design.

Withings Go activity monitor with 1.1-in. circular, segmented e-paper screen.

Assuming that customer demand drives the adoption of consumer devices, once the technology to realize these is available, we can infer from the exhibits shown that there is a demand to minimize the bezel and dead space in a watch form factor display. Companies are striving to provide a bezelless design similar to what has become possible in mobile phone displays. This is much more difficult using a round shape. AUO showed in two symposium presentations how this can be done using a plastic substrate display. Instead of placing the driver chip on the face of the display, in a ledge, or using a TAB lead, they bend the flexible substrate itself to place the driver at the backside of the display. This way a bezel of 2.2 mm can be achieved, with clever gate driver placement and bringing the power lines into the active area from the opposite side of the display face.

Another direction in the development of wearables is to introduce a band form factor display that wraps around the user’s wrist. Canatu, the Finnish touch panel maker, had an E Ink based display device from Wove on its stand. 

Wove wrist device with Canatu integrated touch system.

The touch panel was assembled in an “on-screen” touch fashion to make a complete, integral structure without any separate outside encapsulation. The whole module thickness is only 0.162 mm, according to the press release.

So, it seems like the technical capabilities in displays are coming to terms to satisfy user needs in wearable devices. With the round-faced and band-shaped form factors making it possible to wear a watch again, the “Internet of Designs” can begin. –Jyrki Kimmel for Information Display

Orbbec Shows New 3D Camera Technology in I-Zone

Orbbec Technology found its way through the rigorous committee selection process and into the I-Zone this year at Display Week. The Shenzhen, China, based company has 3D camera technology that Business Development Manager Agnes Zheng claims offers higher accuracy, lower power, and easier connectivity to more operating systems than the flagship Microsoft Connect II. Zheng has a masters in mechanical engineering specializing in optical measurement. She was part of the group that spun out of a university research project with its IR laser sensor technology that she claims can measure objects at 1 meter with accuracy levels at 1-3 mm in measurement of size and distance to the object.   

"The laser sensor we use has a narrow bandwidth laser light that does not get absorbed by dark surfaces. We designed it in-house and have it specially made for this product," Zheng said. They also added an improved bandwidth filter and improved algorithms, all contributing to the higher accuracy performance. 

The group has support not just for Windows,(it's Windows exclusively on the Connect II), but also Android and Unix platform development. It also will sell an OEM module for individual product design projects and integration into multiple devices including LCD- and OLED-based TVs. Power is another advantage over the popular MS Connect II, as the Orbbec 3D camera runs off a standard USB2 connection with 1.8W maximum draw, far lower than the MS 5.0 W requirement. 

The retail version of the product is $150 and requires no power adapter, putting the Orbbec at parity with Connect II when the price of the external power adapter is added on to the $99. 

Microsoft moved to a time of flight (ToF) model in the Connect II while Orbbec uses a unique dot pattern the company designed using the structured light approach. Zheng told us Orbbec has global patents on this technology. 

Meanwhile, back in the I-Zone, users had a blast using motion detection to control the Sony HD flatscreen, playing real-time games and showing off just how accurate 3D gesture recognition can be. -- Steve Sechrist

Nano Composite Materials Empower New Wave Guide Breakthroughs

Display Week's Best in Show small exhibit winner DigiLens is looking to do no less than change everything about how AR/VR interfaces with humans. 

DigiLens had a killer demo that was one of the most popular at the show -- you got to check out their technology from the back of a BMW motorcycle. 

The group, founded by Dr. Jonathan Waldren, CTO, says its holographic optics based on the new composite material solve the latency issues around eye tracking with the company's switchable Brag Grating approach (as opposed to surface relief grating.) It delivers a ground breaking 40-degree field of view spec (versus 25- to 30-degrees using conventional methods) with an upside potential for 50-degrees by the end of the year, and up to 90-degrees in the future. 

A specially equipped motorcycle helmet served as the interface. 

Waldren showed us his version of the future, in which a person's gaze is constantly being tracked by a non-intrusive AR or VR system, feeding that data to the system at very low latency. "This is early days, think Steve Jobs in the Xerox PARC lab seeing the mouse interface for the first time. 

The nano materials breakthrough "...allows development of holographic system with an 8X higher index," Waldren said. The coming world of AR/VR (augmented and virtual reality) will gain an immeasurable boost from a low latency visual system that both delivers the image and knows exactly where the user is looking. The display not only shows content, but discerns user intent, augmenting and responding in a natural hands-free mode. And just as the mouse empowered a whole new graphic user interface, decades ago, reliable gaze and eye tracking technology has the potential to change everything yet again. -- Steve Sechrist

Internet of Wristbands

The Internet of Things is here, as experienced in the SAP Center in San Jose during Display Week at the pregame lightshow. The spectators were handed out wristbands with LEDs, synchronized via a radio interface. The wristbands also had a motion sensor so they would light up in Sharks teal when shaken about. The audience participation helped the Sharks to a 5-2 win over the Saint Louis Blues! –Jyrki Kimmel for Information Display

Spectators at a San Jose Sharks game create a light show with synchronized, color-changing wristbands.

Sensor Integration Drives Mobile Display Evolution

Whereas large, high-definition displays usually get people's attention, there is a diminutive class of displays that virtually everyone takes for granted. Mobile phones have interactive displays that a vast majority of SID show attendees use daily, without giving them a second thought. The quality of mobile displays today rivals that of TVs and in many parameters exceeds them. Some key trends in mobile displays were highlighted in the SID Symposium keynotes as well as in the introductory talk for the Market Focus Conference on Touch. 

Hiroyuki Oshima of Japan Display Inc. (JDI) gave the conference keynote on mobile displays, highlighting JDI’s strategy to concentrate on core technologies. One of these is an in-cell touch-based user interface. Other core technologies from JDI include LTPS and IPS, which support the touch functionality that will take on new capabilities. JDI sees the future growth for display business in new applications as the mobile phone market saturates.

A mobile display from JDI (photos by Jyrki Kimmel). 

Calvin Hsieh from IHS gave the lead presentation in the Market Conference on Touch. In the IHS forecast, in-cell touch for AMLCD and on-cell touch for AMOLED play a large role, shown in projected growth for these technologies. For touch in general as well, new applications drive the growth of the business.

What new applications are there for mobile displays and for touch technologies? A lot of these rely on sensors that are being integrated into the module itself. These sensors give the mobile display capability for multimodal user interaction, from fingerprint and proximity sensing to hover touch. These interaction modalities can then be leveraged over a wide range of application areas, even in automotive use.

Another trend is the proliferation of organic form factors in small and mobile displays. Sharp comes into this area from another direction, taking the form language from its automotive curve-edged displays and transforming mobile-sized displays from rectangular to round and oval-shaped objects. 

Sharp has been specializing in "non-traditional" display shapes.

These new form factors, combined with curved display integration, led by Samsung, open a way for totally new device classes, beyond the mobile phone and rectangular, passive information screens in cars.

Samsung is also experimenting with some interesting form factors. 

From the presentations in the conferences and modules shown in the exhibition booths, it seems that the predicted curved and flexible displays are still as far in the future as roadmaps depicted a few years ago. Prototypes of “rollable” screens are becoming ubiquitous but real products are still beyond the horizon. Until we get there, there will be many advancements in “classic” mobile display technologies that in turn can propagate to other application areas, making developments in mobile displays the vanguard of evolution in display technology. Sensor and system integration as well as touch user interface evolution will play a major role as constituent technologies in this development. –Jyrki Kimmel for Information Display