I imagine it has to be in the 8 to 10 inch size range, touch-screen, WiFi. In terms of Industrial Design I expect it to look in between a Macbook and iPhone. I expect it to run MacOSX and for there to be the standard productivity apps we see on the iPhone, with a very well executed eReader app. A portfolio application like OneNote would be nice. One thing I know is that it will be accessible – Apple has worked hard over the last five years to ensure the Mac, then the iPod, now the iPhone are accessible. A tablet has great potential as an accessible device. At a minimum the screen magnification and text to speech capabilities in MacOSX on a tablet could be winners for people with low vision. The tablet form factor and large touch screen open up a world of input potential for people with motor and cognitive challenges.

I’ve used a few tablets and got one of the first Kindles, I’m no stranger to the form factor. What would make a tablet from Apple a hit? I think it will be about the content. I’ve long advocated that the introduction of micro-payments would lead to the creation of quality online content. I think Apple has the infrastructure and the audience to support this. Have a way for me to pay tiny amounts for the content I use and thereby open up a new paradigm in online publishing. I’d be very happy for a $10/month subscription to be shared among all the people who provide me with content. If Apple really pulled this off we’d see the emergence of new magazine, weekly news, newspapers, etc all designed for this form factor. That’s something I’ve been waiting for a long time.

What are the wildcards? What would delight me? An option to use a stylus (Inkwell), to dock it vertically for charging and connecting a keyboard and mouse.

And if all this does not come to pass, I’d be very happy to see Apple ‘just’ make a great eReader that sits in between my MacBook and iPhone and gives me access to my iTunes content, news, blogs, web, books, etc. Let’s see what comes to pass …

The Deaf-blind communicator connects with bluetooth to a smartphone, enabling a deaf-blind person to communicate with a sighted person anywhere..

Richard: Jim, I wondered if you could talk a little bit about the most recent product you’ve worked on – the Deaf-Blind Communicator. From a design point of view what were the challenges?

I should first explain what it is. The DBC was designed for people who are totally deaf and blind to communicate and when you think about those disabilities it’s very difficult to communicate with the mainstream because most people don’t read sign language. Even if they could read sign language, to communicate back to the deaf person if they can’t see the signs, then you end up having to sign into the person’s hand. So two-way communication is extremely difficult. In the past they’ve had a technology that was produced in the late 1980s that had a TTY capability, which is a very old kind of almost Teletype communication. Slow speed, one word at a time as you type it out, you could only communicate to someone else who had a similar device and could read the text output on a small LED display. Deaf people had communicated that way for a long time over the telephone. But deaf-blind people had a problem with that because they couldn’t see the text showing up on the little LED display. So a Braille display was designed so that they could have that converted into Braille and read the Braille one line at a time as it came across. That was pretty much the kind of communication that existed for somebody who was deaf and blind and what was required was the ability to continue doing that but also to communicate face-to-face. If you wanted to go to the grocery store, if you were in a taxi or on a bus, or in a restaurant, or something like that, how does a Deaf-Blind person communicate? It’s extremely difficult for them to do that.

So, we looked at what was needed and we decided we had to have a device that performed those basic functions – the TTY functions, and the face-to-face function. It had to be a portable device though, because DB people are usually carrying a bag of other things with them, they may have a dog, they may have a cane in one hand. They can’t carry around a whole lot of stuff because they’ve got a backpack they already use. So there’s a practical element and we did a lot of research trying to figure out what they were willing to carry. At the same time what did the sighted or hearing person with whom they were trying to communicate require. That took a little while to work through, and luckily we’re in an age of technology now where smart phones and cell phones and Blackberries and things like that exist. So the general public out there is used to seeing these small devices, seniors may not be using them necessarily, but the average person in a restaurant would or a taxi driver might. So we finally narrowed it down to having a portable device you could carry with you. We also wanted it to be off the shelf because the population here is not a huge population so you can’t sit down and design a product from the ground up and produce all the molds and all of the circuitry without it costing $20,000 per unit or something! So, we picked a cell phone as the external device for the sighted person to use to communicate with the DB person.

Then we had to decide what is the DB person going to use. The DB person needed to be able to use Braille because they couldn’t see and they couldn’t hear, so you couldn’t use speech. So they had to be Braille users. And again the device had to be portable. There are a number of DB people who are only communicating in a TTY fashion and they need a face-to-face fashion. Ironically, in this day and age, we have a technology that fits very much into the way Deaf people have communicated for a long time – texting or instant messaging. Those sorts of things are happening in the mainstream now and people text each other all the time rather than calling on the phone. So the irony is that it’s almost the kind of communication that deaf people are used to with TTY. So if you have a device that has the capability of doing some of those things and producing it in Braille – so whatever I type on my keyboard, whether it’s a Braille keyboard or a standard qwerty keyboard, gets sent out to the other device or as a message to anybody’s cell phone or TTY depending on who you are trying to communicate with. But when you do it that way all the conversions happen and the translations happen automatically. The same thing when somebody sends you a message back, the device has to be able to do the translation within the device and produce whatever was sent to you in a Braille form. So HumanWare produces a product called the BrailleNote, which is probably the most broadly used portable Braille PDA around the world. It’s really a well-respected product that thousands of blind people are using very effectively. So we thought if we could use the BrailleNote as the platform we could simplify the user interface for those who only wanted to use face-to-face communication on the TTY. But we could also have a menu item that had advanced features and if you went into that and you activated the advanced features suddenly you’d have a full-blown Braillenote. You could do email on it, you have a full word processor with a spell checker, a scientific calculator, a database manager, address list, GPS, and all kinds of applications. There are all kinds of things that you could have with that device that the mainstream has on a smart phone or a PDA. So we thought here’s a way of creating a device that a DB person could communicate with and the person at the other end would never know, unless they were face-to-face, that the person is DB at all. When you’re typing on one these devices its not super fast. However if we’re in a meeting situation, where someone is giving a lecture and the DB person wants to hear the lecture you could plug in a USB keyboard directly into the Braillenote and just type on it and that would instantly come through in Braille. So you could have a pretty fast typist on that keyboard where they wouldn’t necessarily have to use the little smart phone keyboard.

The phone also provides a couple of other features. Because it vibrates, if a message comes through – if you’re getting a text message or a TTY it actually vibrates so you know you’ve got a message.

Richard: How did you involve users in the design process?

That was important because the population has such a range of capability. There are DB people with PhDs who are working full time and who use computers with Braille terminals attached. Then there are people who’ve never used any technology whatsoever. There is a population who has Usher syndrome. Typically the person goes deaf before they go blind so they have what’s called retinosa pigmentosa which is an eye condition that has tunnel vision which slowly but surely gets smaller and smaller until it disappears. But those folks have a tendency to want to use their vision as much as they possibly can but because retinosa pigmentosa is going to result in total blindness they really should start learning Braille at a very early stage. They don’t always do that, so you have people who are just learning Braille and they’re not used to it and so because the vision part of it goes maybe in their forties or fifties that part of our study group had to be taken into consideration too. What if they don’t know Braille yet? So it had to be as simple as it could possibly be, for people who had never had any use of technology whatsoever, to higher end folks who could pick it up pretty easily and may want to have all of the other features of a PDA. We had to include all of those in our initial tests to see are we simple enough and yet are we satisfying the needs of the others. From that we realized that the hard part was going to be the non-techie people, who had to communicate but couldn’t grasp anything sophisticated. So from that we decided we had to almost create two devices – a shell type device that is about simplicity that just does the most basic things and start with that. And we had to create some training that was just geared towards that. The people who were able to use email and the more advanced features have generally had some sort of access to a computer before. So we decided if we could cut off all the advanced features and somehow activate those for advanced features that would work. So they wouldn’t have to sit behind the cockpit of an airplane if all they wanted to do was ride a bicycle.

Then we brought people in to try it, to see if it would work. Usually people in these test groups have opinions they express, but I think the opinions we received in these test groups very much related to the sophistication of the user and we realized it is very hard to create one device that is usable by everyone, so we ended up with the lowest common denominator as the interface and gave them the option to activate the rest.

We also noted that training was going to be an issue. One of the challenges there was that if you have someone trained up you wanted the trainer to become their resource rather than the company, because the company was in no position to do support with end users. It’s a very time consuming thing and it’s an expensive proposition. Our goal was to have trainers in every state and for those trainers to have access to tech support but that the end user was working through the trainer. People who understand the DB population and know how to provide the trainer.

So it wasn’t just about trying to figure out the product and make it simple, but also ensuring how there was some training and support that would be part of the solution.

If the end user was sophisticated enough to access the advanced features, then that user is probably able to use our regular tech support directly and work through email the same as a regular blind user.

Richard: How has the reaction been?

I think it’s been very positive and I’m very excited about it.

Richard: Are they able to use it and do things they couldn’t do before?

Sure, as an example they’ve never been able to go into a Starbucks and order a coffee before and now they can.

Richard: That’s a challenge anyway – some people’s coffee orders can be quite complicated!

There are certain functions you want to repeat. For instance you get on a bus and want to tell the driver what stop to get off on, so why not program that message. I want to get off at so and so street, most people order the same drink every day, maybe at a restaurant it might be more complex. For a lot of advanced users many restaurants have their menus online, so you can sit in the restaurant and with a WiFi connection you could read the menu on the web, then order it. So you have the ability to do a lot of different things with this depending on your level of sophistication.

Richard: How is it for the other person in the conversation?

That was very interesting because obviously the person on the other side of the conversation would need to know how to type and spell. If you go into a bank then the conversation is pretty smooth because everyone there is used to using a keyboard. The problem is if you pull something out of your pocket and hand it to the teller alarms can go off! So there has to be a little careful of those settings. But if you go into a McDonalds that person may be able to take the order and push the buttons but they may not be very good at typing. So in those cases we found that a manager may come up and do it. In one case on a bus the driver was an older guy and had arthritic fingers and he just was very concerned about typing on this small phone. There was someone behind him on the bus who said let me do it and so they jumped in. The same thing happened in a convenience store where another customer was getting frustrated waiting while the checkout clerk was having trouble knowing what to. The other customer saw what was happening and they jumped in and started to explain to the person behind the counter what he needed to do to type into the thing. But in every case we found that the communication happened. Sometimes because someone jumped in to help but in every case the DB person was able to communicate what he or she needed. It’s not perfect but when you think about not hearing and not seeing and being able to travel through the mainstream world it’s a pretty amazing solution, it completely frees you up. So it completely does away with the disability in that sense.

Richard: What have been the high points of your career as a designer of products?

I don’t know that I’ve ever really been the designer; it’s always a team effort. Certainly the DB project has the most freeing and life-changing impact of anything I’ve worked on. Also I’d say myReader is another important product for people for with low vision because it is much more of a reading device. A traditional CCTV is really slow and not an easy contraption to use. It’s more of a point reading device, whereas myReader captured the information and allowed you to present it in a number of ways so that you could have large print up on the screen but you’d have word wrap and you could read it like a teleprompter and have a whole page come across on a horizontal line. That meant that rather than using a table that moves underneath a camera where you might have 150 lines of text in a magazine article that you’re trying to read and you have to move the table back and forth 150 times and hold your thought at the end of every line go back and find the beginning of the line. The speed of comprehension and the fatigue of all of that would go away as soon as you able to reprocess and present all of that data in a single line of text. For me that was another product that completely changed the way people read.

Richard: Any other products?

I think the Braillenote when it was first introduced in 2000 was an amazing device. There had never been a real PDA for blind people before, there had been basic notetakers, but as the rest of the world started doing email and started communicating in other forms, having a basic notetaker that was pretty limited in terms of its word processing and didn’t have a real database. That was all possible on a Braillenote, and it completely changed the way blind people communicated with the mainstream and blind people were at an equal level again. For example if you had written something in word and wanted to give it to them they could open it on their Braillenote and vice versa. They could hand you a word document, which they had completely written in contracted Braille and back translated and you’d never know it had been written by a blind person. So the disability goes away. I think if anything that’s the message on everything I’ve worked on, if you can equalize the means of communicating that’s the goal. So with the Braillenote thousands of blind people are competing in regular jobs.

Richard: So the technology becomes the enabler?

Yes, it unleashes the person’s innate ability to perform without having the disability get in the way. We make up for the disability and compensate for it and they can have a freer life.

Richard: What would you advise a young person getting into the disability field?

You need to familiarize yourself with the people you’re trying to serve. Understanding what’s out there and what’s not out there, sometimes it takes fresh mind and ask “why haven’t we done it this way?” There may be a good reason, but maybe nobody has ever thought of it before. In our industry I think we often get into a ‘me too’ way of thinking – where you just do the same thing over and over – maybe it’s a little smaller or faster. So it becomes an industry of technology rather than solutions to a given problem. The CCTV is a great example, it has been around for 35 years and it’s the same. Ok maybe its color now and maybe the xy table moves more freely, its not even much cheaper. We’ve been rehashing the problem. If people come into the industry they see the problem is not to design another CCTV but to help people to read. So, you have to ask yourself where do people normally read? I read in my easy chair, I read in my doctors office, I read on the bus. There are all these places that I read. The CCTV is a solution, but a it’s terrible reading solution in my opinion, but thank goodness it was there because it was the only one we’ve had. But then you think about what the alternatives could be. When you think about reading, you don’t necessarily want something where you assume a rigid position in front of the device. Reading is something where you lounge, you kick your feet up, you lay your head back, you get in a comfortable position, and you get lost in your reading. So you want to have something that accommodates your natural desires to read, instead of re-creating a device that’s already available. Ok so there are the basics you have to have larger print, you have to have great contrast. But we really need something that’s not a CCTV.

Richard: So the task is really to understand the task and build a device that supports that, rather than build a better CCTV.

Exactly, its one thing to produce a device with large print, its another to say well where is the content that you’re trying to read, where does that come from. If you think of that, look at the Kindle – one of the things that make it successful is that the content just shows up. So you have to think about how do I get my content and how do I make it accessible to me. Then the solution is something that puts those two things together and allows me to read wherever I want to read. That’s a whole lot different from designing another CCTV. I think there are wonderful opportunities out there for technologies that nobody’s ever produced. Interestingly for the companies that already exist out there, it’s a risk to put money into developing a new technology that may not work or that may not sell. So if they spend two or four million dollars on a development and it doesn’t work then the company is out of business. If you have investors, and in this day and age that’s tough, particularly in the disabilities area it’s hard to find investors anyway, so they’re really averse to risk. So you end up seeing occasionally something like a Braillenote come out, occasionally something like a myReader, where you’ve got a whole new approach to technology. In the case of the Braillenote that was a risk that paid off, it turned the company into a whole other level. So sometimes the risks are worth it and sometimes they’re not. But I think the greatest risk is to keep doing the same thing over and over and ignoring what the actual customer needs and wants are.

Richard: Thanks for sharing your thoughts!

Reflecting on the discussion with Jim, I think there are several pointers for developers of assistive technology products:

• Don’t be incremental – understand the problem you’re trying to solve. Take a fresh approach; don’t just design a better version of the current products that are out there.
• A user-centered approach pays off – involve the people who will use the product in its design – have them try prototypes, you’ll learn a lot from this.
• Build a solution – remember people are looking for a solution to their problem, not a product. In the case of the Braillenote, myReader, and the Deaf-Blind Communicator these products provided a solution that was beyond what current products offered.
• Try to use off-the-shelf products or platforms – it’s tough to get the capital needed to invest in custom products. Where possible adapt existing products or platforms.
• Use the technology to make up for the disability – the goal is to use the technology as an enabler, so the disability ‘goes away’.

For further details on how the Deaf-Blind Communicator works, check out HumanWare’s product site. A great presentation on the deaf-blind communicator by Greg Stilson is available courtesy of AccessibleWorld.

The building is also an interesting story in green design – more on that in another post.

A rendering of the New Zealand ICT Innovation Institute building at the University of Canterbury

The Institute has a really interesting value proposition combining Ideas, Interaction, and Investment focused on industry-relevant themes:

The accessible ramps in sidewalks make the sidewalk easier for all of us as well as people in a wheelchair. In the same manner VoiceOver makes it possible for blind and low vision users to navigate the iPod, but this same user experience is great for people who don’t want to have to look at a screen – for instance while exercising.

The audio interface is controlled by a three button remote molded into the right earbud cord. Two buttons are dedicated to Volume Up and Volume Down. A third button is used to control the VoiceOver interface as follows:

• Single click – Play/pause.
• Double click – Go to next track.
• Triple click – Go to previous track.
• Press and hold while a song is playing – Hear the title and artist. 
• Press and hold until you hear a tone – The playlists will be listed. Click when you hear the name of the playlist you want to go to.

Apple iPod Shuffle with small controls for VoiceOver user interface on the right ear bud cord.

Louise Braille was born into a family in France in 1809 and grew up 25 miles outside of Paris. His father was a Bourrelier – a harness maker. At this time, the steam engine had been around for a while, but horsepower was the predominant force in transport. So, Simon-Renee Braille had a very useful trade and his own small business, providing leather harnesses to local farmers. It was in his father’s workshop that three year old Louise damaged his right eye while trying to emulate his father’s craft with a Serpettte knife. The damage to his eye and an auto-immune reaction in his right eye (Sympathetic Ophthalmia) led to blindness at the age of five. Louise was educated at the local school and then at the age of twelve left his family and moved to Paris, where he attended the Institution Royale des Jeunes Aveugles – a school established by Valentin Hauy.

Hauy was an inspired educator who believed blind children could learn. In 1784 Hauy had observed a blind student Francois Le Sueur recognize the letter O which had been indented into a printed invitation. The letter was slightly embossed because too much pressure had been applied on the paper by the printing press. As with many inventions, Hauy’s observation was a serendipitous event which would contribute to a path of invention. Hauy developed a method of printing which produced embossed letters in paper – raised up letters that could be felt with the fingertips. There were a lot of details to work out – the metal type font had to be reversed so the raised letters were not mirror imaged, special paper developed that could be dampened and embossed without tearing, and a method found of preventing the paper from sticking to the metal type. By the time Louis attending Hauy’s school in 1821 the children were actively being taught to read this embossed paper with their fingers.

The next enhancement was a new approach to embossing developed by Charles Barbier, an ex-artillery officer. Barbier had developed a special code which could be used to make notes that could be read by touch in the dark. The code was made up of two columns of six dots. The dots corresponded to a six by six matrix of 36 syllables, vowels, and consonants needed to approximate the sounds of the French language. Using the number of dots in each column as a reference for the rows and column of the matrix, each raised dot pattern corresponded to a sound. The military were not very interested in Barbier’s ‘night writing’ system and so he advocated the potential of his invention for use by blind people. In 1821 Dr Pignier, the Director of the Institution Royale des Jeunes Aveugles, agreed to try Barbier’s system with a group of students. Louis Braille, at that time 12 years old, was one of the students.

There were problems with the system – it used a lot of space, and the dots did not fit under the fingertip, and it was not a direct parallel to regular French text. Over the next three years, inspired by the concept of the embossed dots, and already a user of Hauy’s embossed letter system, Louis developed his own technology. Louis reduced the number of dots to six and laid them out in two columns of three. He then mapped the dots to the letters of the alphabet rather than to sounds. By the time Louis was sixteen he had perfected what was to become known as ‘Braille’– the system of raised dots that is used all over the world to denote language, music, and mathematics.

The correspondence between Braille code and English alphabet.

The remarkable thing about Braille is it enables blind people to be literate. Two hundred years after the birth of Louis Braille, many people’s lives have been enriched by his technology and for that Louis Braille is deservedly internationally renowned.

Today, braille can be formed electronically and matched with a computer to produce ‘refreshable Braille’. Using devices with this technology, blind and deaf-blind people are able to access print from many sources. Accessible websites can be turned into braille so users who are unable to see can read the text on the pages. The trick is to make sure the content has been developed according to accessible design principals evangelized by the World Wide Web Consortium WC3 in their Web Content Accessibility Guidelines. As we celebrate Louise Braille’s two hundredth birthday, let us commit to following these guidelines.

This post draws on a fantastic book called ‘Louis Braille – A touch of genius’ by Michael Mellor, published by National Braille Press who makes it available in print and braille. Mellor does a great job of describing the life and times of blind people in the 1800’s and goes into the history of Braille in great detail. He publishes for the first time personal letters written by Louis, which provide a fascinating insight into his life. The National Braille Press has also developed a great website which celebrates Louise Braille’s two hundredth birthday. Happy birthday Louis!

‘You have to promise me that if this initiative is going to fail it will FAIL HUGE’.

Well Alan and his team did not fail, in fact they succeeded in making an industry more aware of the potential the computer had to change people’s lives by empowering them.

But this book is not about what Alan has done in his career, that is really a small part of it. 

The book is full of great observations about the assumptions we make as to what it means to be disabled. He manages to challenge us and at the same time leave us inspired to do the right thing – to be enablers. I encourage you to read it – this is the sort of book that can have a profound influence on your thinking and also give you hope for how things could be. Alan continues to lead the charge on these issues at Yahoo. Meanwhile, the people at Apple are doing a really good job of stepping up to the continued challenge of making their products work for everyone. Also check out www.disabilityland.org