Me @citizenbeta

I spoke at Citizen Beta this week. It was lots of fun and I wanted to highlight a few things.

(I really love this photo)

 

The topic of the talk was AAC in general and the AzuleJoe project in particular.   I wanted to make sure that I was giving the audience something to think about (rather than it being a recruitment drive) so we included some of the privacy and free speech issues that  turned up here.    There is also a lot of heavyweight civic tech in the room so we also talked about the use of freedom of information request in the process:

 

 

To be entirely transparent, much of my motivation for going was to recapture a mindset.  I’ve been effectively working in the third-sector for two years and I have relatively few conversations with people who think like me. Citizen Beta is full of people who do and it was lots of fun to simply hang out and find out about the cool things that people were doing.   I spoke to people who were verifying news, making digital payments easier, a guy who was organising a strike of renters, someone writing a book about digital revolutions and a wide range of people who where doing That Sort of Thing.

I want to do a suitably big shoutout to Mevan for putting me in the room, and to everybody else I met on the day.

 

On consent in AAC user testing

We’re in the process of running some user-testing for CommuniKate and AzuleJoe.   User testing is an interesting proposition for the sort of things that we are building given the extreme ranges of situations that people in AAC deal with.

 

User-testing is that thing that designers don’t want to do because they know their design will work.  And user-testing just wastes time.  The world is littered with the bones of people who have that view.

We’re using our user-testing as a chance to get to proper grips with where some of our users are, with the issues that they want to talk about (often different from the ones that affect them), and with the stories that they want to tell.

 

Time allowing, we’ll talk properly about the user-testing itself in a later post, today I’d like to show off our information sheet for the study. Kate made it, and it is a work of art in its own right.

 

Screenshot 2015-11-16 20.30.13

We want to make sure that our information sheet is a real one, and it genuinely gives of all the information it possibly could do for users with cognitive issues.

Of course, aspects like data retention and anonymity are difficult to pull off.  Kate and I went back and forth quite a bit on a suitable symbol for ‘anonymity’ (more on that in a future post).

Screenshot 2015-11-16 20.39.52This is the first draft of our information sheet. You can view the full on in Word and Pdf, we’d be interested in any comments people might have.

 

AzuleJoe Hacking

Table shot

The purpose of this post is to show you how easy it is for a hobbyist to retarget AzuleJoe with nothing but Paint and Notepad.

Motivation

The injection of the Inclusive Technology Prize money has meant that I have been able to spend a lot of time recently developing on AzuleJoe – it’s moving forward on a variety of fronts and there will be a couple of proper announcement posts in a few weeks.

This is, obviously, fantastic.

On the other hand – it’s also making it a little less ‘hackable’. One of the joys of the original code was that it was easy to retarget – if you had a bunch of pictures of grids and a text editor, then you could retarget AzuleJoe to talk about anything you wanted.

In recent months the code has started to look a little more professional – we use JSON for the data rather than Javascript, and quite a lot of extra code has been added to do all sorts of other things.  Our next big push will probably obliterate most of the original code anyway.

I think this is a bit of a shame, because I’d like to keep that ‘hackability’ as available  as possible.

The code

In an effect to keep an easily ‘hackable’ version of the code around. I’ve released a ‘tagged’ version on the github repository: https://github.com/joereddington/azulejoe/releases/tag/v0.9 (at the time of writing there have been 69 commits to the repo since the release, so the code is moving on without it).

The Demo

To show how easy it was to retarget the software to any pageset, I prepared a three minute video (bits of it are sped up – it was about 10 minutes in real life)

The voiceover was recorded while I was sick with cold and the mouse movement and even the typing is difficult to follow at times, but the point is that it points the curious in the direction of the right files to play with. And I’m on the other end of email if anyone would like more info.

 

Inclusive Technology Prize finalists!

 

Screenshot 2015-07-08 12.00.34

Kate and I are in there somewhere!

 

Bit of an announcement – AzuleJoe, has been chosen as one of the ten finalists for the Inclusive Prize (we’ve known for a little while, but the embargo has only just been lifted).
We’d reached the semi-finals a few months ago, I didn’t do a post on it at the time for a few reasons – I didn’t feel it was interesting enough for the readers, and it was a relatively self-centred thing.  More to the point, I was still getting my head around the business aspects of the prize and hadn’t got a far as a clear vision about how everything fitted together.
Kate and Joe holding a tablet in front of an 'inclusive technology Prize' sign

This is us showing off something that I haven’t even mentioned on the blog yet… *blush*

I have now reached that point and we have a clear view on how we can keep our development open source, ensure that it’s free at the point of delivery, and user-focused, while still presenting strong business outcomes.  More on that in a future post.
As a sidebar – we were slightly unusual in avoiding publicity at this stage – because of my position as a blogger I was getting a few press releases from (to be fair, media departments that worked with) other competitors about their work and how the’d reached the semi-final – although to be fair, everyone who sent a press release also made it to the final…     
A lot of the other semi-finalists do have extremely strong ideas and I intend to be featuring some of them on this blog in the near future. I’m going to start with my favourites of the ones that didn’t make it to the semi-final so that there is no suggestion of competitive scheduling 🙂
The final is slightly different. For a start, it’s one of ten, rather than one of 25, and for another thing, finalists are given £10,000 and, equally importantly, lots of help and guidance in the following eight months to develop to a full prototype.  This money is desperately needed to help develop both AzuleJoe’s technologies and CommuniKate’s reach.  The development day Kate and I went to yesterday was fun and interesting – we chatted to Justin Tomlinson the new minister for disability, and we worked with lots of cool and interesting people to really nail down some of the ways to move forward.
It’s also the biggest grant that eQuality Time has ever received.  This means excellent things for eQuality Time.  Not only is it definitely going to be in operation come April 2016 (which, as you’ll remember, is almost twice as long as we expected it to be running when we started), but it’s got budget, and guidance and a clear vision going forward.
As always, there best part was meeting the very cool people there… This is me getting the opinion of Open Bionics on a 3d printing project that is completely separate to either of our projects but that should be part of the blog sometime soon…
Screenshot 2015-07-08 12.01.27
…and us having a really good chat about going forward with handy experts from Leonard Cheshire Disability (I know it looks like we were the only people having fun, I think that is just a trick of the photo, I’m assuming everybody else was having fun too… )
Screenshot 2015-07-08 12.03.09

CommuniKate, AzuleJoe and the Awesome Foundation.

Screenshot 2015-03-17 17.43.38
The idea of launching CommuniKate as the first open licensed AAC page set was conceived exactly here, when Kate and I had dinner after BETT2014 (you can read my opinions of the event itself here).
So it was kind of nice that, exactly one year later, that Kate and I were invited to pitch CommuniKate to the Awesome Foundation.

The Awesome Foundation

If you are unfamiliar with the Awesome Foundation it’s worth learning more. Here’s the blurb from the website.
The Awesome Foundation is a global community advancing the interest of awesome in the universe, $1000 at a time.
 
Each fully autonomous chapter supports awesome projects through micro-grants, usually given out monthly. These micro-grants, $1000 or the local equivalent, come out of pockets of the chapter’s “trustees” and are given on a no-strings-attached basis to people and groups working on awesome projects.
(The UK version is £1,000 rather than $1,000)
The pitch events are wonderful.  Kate and I pitched alongside Board of Media, a performance poet, people with a concept for reusing food and Soundcastle. For most of the evening you are mingling with these guys and the trustees (who are all people who’ve made very cool things in their own right). So you have an evening of chatting to driven, open, wonderful and emphatic people.  In every conversation you find yourself saying “oh I know a guy who can help you with that” or people saying “Try Fund X, they are interested in what you do”.  To paraphrase Terry Pratchett “there’s nothing like being near happy startup people, it’s like giving your brain a hot bath”.
You absorb all the passion, drive and energy;  it really lifts you.

The Contest

After all that, the contest element appears a little bit separate… you’d certainly go home happy if they just didn’t bother to announce the winner.
Luckily, they did announce a winner, and even more surprisingly it was CommuniKate. (and it really was surprising – no false modesty here, the other groups were good).
So we had a bit of a party.
Screenshot 2015-03-17 17.44.03

The Speech

For two reasons, firstly because we like transparency, and secondly because I like the speech, I’m putting the speech I gave to the evening here. It was very much under-rehearsed on the night and most of the best interactions of the evening were unscripted.  I like to think the pitch clearly sets out the problem, what we want to do to solve it, and what’d do with the money.  Beautifully, when the trustees announced the awards they said: “We’re giving you the award, but we don’t think you should spend it on that.”.
 
So. This is Richard.  He has severe learning difficulties, many physical issues, and behavioural issues that often result in violence.  
 
He’s also my little bother.  
 
Richard can’t speak or read. Until he was about 19 we managed on maybe 90 signs, mostly nouns.  It’s worth considering what your life would be like if you only had 90 words.  
 
Then, when he was 19, he got this.  It’s an AAC device.  AAC devices work like this: 
 
(Show this video) 
 
 
Suddenly Richard has 5,000 words,  it ab-so-lute-ly changed his life.  
 
The problem is that Richard isn’t alone. There are 32,000 people in the UK that would benefit from powered AAC.  Unfortunately there are only 9,000 people who do.  That means that there are 23,000 people who can’t choose what they want for dinner, can’t tell a doctor their symptoms, can’t tell their family that they love them.  
 
If you are wondering why do these people not have access.  This device was £5,000 at the time.  I saw devices at BETT today that were £15,000.  Communication is priceless, but there are families who have to choose between heat and food right now.  
 
So last year Kate, who has worked in AAC for about 15 years, came to me and said “I have a page set that has been working really well with my clients – I want to launch it worldwide, can you help?” and I said “Let’s talk about open licensing”.  
 
So ten months later we launched CommuniKate, a completely free, Creative Commons licensed page set that people could use on their own hardware.  Complete with manuals, online versions and testimonials. You’ve actually already seen it in this video. The response was amazing.  We’ve now got volunteers working on the code to improve it, and we’ve got teams translating it into Portuguese, Spanish, French, Swedish, Urdu and Arabic.   Kate had people she didn’t know the name of hugging her in the street.
 
I can give you a link, and you can play with it yourself inside of 20 seconds.  We think this is going to make a difference to thousands of people in the UK and we hope it’s going to make a difference to many more outside.  
 
So here’s the problem.  Free text to speech software sounds like this:
 
(play first recording) 
 
   and doesn’t support languages like Urdu and Arabic. 
 
A decent commercial service would sound like this:  
 
(play better recording) 
 
and it would be much faster.
 
But a decent commercial service and all of the licensing for this sort of thing costs around £700 and we don’t have that money.  We were hoping the awesome foundation might be willing to help.  
 
 
 

CommuniKate: Vision

So, The CommuniKate Project is about one year old, and it’s been released about three months now, to much more of a pleasant reaction than we expected.  A few weeks ago Kate and I sat down in Tibbits with the back of an envelope and worked out where we want to go (AzuleJoe has its own goals, more on that in another post).
There are three things: universal formats; different ability versions and, as a pipe dream right now, access beyond hardware.
Screen Shot 2014-10-02 at 12.48.54
Universal formats
Universal formats is straightforward. If you can buy an AAC device, we want you to be able to use CommuniKate 20 on it. Ideally all the devices would work with something universal like the open board format, and we strongly support that. But it takes a while for manufactures to take those things on so we’ll be developing (and looking to help develop) CommuniKate ports to any AAC device you can name.  Giving the users more choice, more security, and easier device switching when they need it is our key goal right now.
Different Ability Versions
CommuniKate 20 is designed for users with complex needs.  Some users have difficulty with the number of items on the page.  In the next year or so you should see CommuniKate 15 released, along with a version of CommuiKate 20 that supports scanning.
Screen Shot 2014-12-10 at 12.00.18
Access beyond Hardware
Something that is always surprising to AAC professionals in the UK is that the UK is world leading on AAC. There are a wide range of factors: the national health service,  our relative wealth as a country, and even more mundane things like certain examples making the whole setup much more acceptable in the UK for many many years.   But the world is a big place and there are people with communication disabilities living all over the world in awful conditions.
We’ve posted about communication books before. We’re going to be investigating the possibility of working with some partners working overseas who might be interested in a copyright-free (translated) way of helping people communicate. For the price of one piece of hardware in London we could ship 500 CommuniKate communication books to somewhere that really needed them.

AzuleJoe: behind the scenes of the CommuniKate 20 demo

One of the parts of CommuniKate 20 that turned out to be very popular is the live demo (you can view a video of it in action here).

It’s time to talk a little bit about that live demo and how it works. This is one of those posts that’s only focused on the technical reader. It will have very little value to the non-coders among you I’m afraid.

The live demo software is called AzuleJoe, it’s entirely open-source and, like almost all the code presented on this site, it’s available to download from github.

When you view the live demo, what you are actually looking at is the CommuniKate 20 pageset running on the AzuleJoe software. The software that became AzuleJoe dates back to 2009 when I built a system for my little brother. When we started to prepare CommuniKate 20 for public release in 2014, the software was significantly overhauled into something that developed it’s own identity

The name ‘AzuleJoe’…

In case you are wondering, the name ‘AzuleJoe is pronounced A-fo-lec-oy, AzuleJoe is a Basque word meaning Tiles.  I Apologise for the ‘out there’ nature of the word but when you are trying to find a word that sounds like it might be something to do with an AAC device and also includes “Joe” as the last three letters (“CommuniKate” rather painted me into a corner) you are pretty happy with a short word that means Tiles.

Why is it separate?

Of course, we could have folded AzuleJoe in with CommuniKate and left it as a simple online demo. The reason we that we separated out the projects should be obvious to anyone browsing the repository: AzuleJoe can demonstrate any pageset, not just CommuniKate 20.   CommuniKate 20 is the first openly licensed pageset, we have no intention that it should be the only. 

If you’ve got a pageset that you would like to see running online easily, then the AzuleJoe code will help out, and indeed, Joe will probably help out if you ask.

 

Design

Some of AzuleJoe’s design features might be a little unexpected on first viewing, but they are generally part of a reasoned view.  Let’s look at one in particular.

Full page images

You might expect that the images that AzuleJoe works on to look like this:

Screenshot 2015-02-13 12.10.12

Instead they look like this:

breakfast

There are sensible reasons for this. First of all it was useful for the development of the pageset. Kate put CommuniKate together as a PowerPoint file. I then exported that into images and put those images directly into the software. Each image has an accompanying JavaScript file that tells the software what do do when each zone on the screen is pressed. Currently I generate these JavaScript files by hand, but the next version of AzuleJoe should be able to generate them automatically from the PowerPoint files.

Here’s the JavaScript file that goes with the image above.

function breakfast(){
reset();
    utterances[0][0]="yes ";
    utterances[1][0]=" ";
    utterances[2][0]=" ";
    utterances[3][0]="";
    utterances[4][0]="no";


    links[0][1]="1";
    utterances[1][1]="breakfast";
    utterances[2][1]="cereal";
    utterances[3][1]="porridge";
    utterances[4][1]="bran flakes";


    utterances[0][2]="toast";
    utterances[1][2]="butter";
    utterances[2][2]="croissant";
    utterances[3][2]="jam";
    utterances[4][2]="muesli";

    utterances[0][3]="boiled egg";
    utterances[1][3]="fried egg";
    utterances[2][3]="scrambled egg";
    utterances[3][3]="bagel";
    utterances[4][3]="full English";

    utterances[0][4]="beans";
    utterances[1][4]="fruit";
    utterances[2][4]="pancakes";
    utterances[3][4]="";
    utterances[4][4]="";

    document.main.src="images/breakfast.jpg";

}

 

You can see how simple this is to build, one image for every page and one very simple file to go with each one.

 

Why is this useful?

We think that pageset design should be as simple as possible for users. If it’s PowerPoint it can be a communication book as quickly as it can be a working AAC demo. We want to be able to say to people “Here is a PowerPoint Template, now you can build a pageset concentrating on the language rather than the buggy interface some programmer threw together. When you are finished. We’ll expand it out for you.”

 

I like open source things, how can I get involved?

We love the idea of people helping out with the code. Because AzuleJoe is currently such a simple project, it’s there are a lot of easy things that people can do to help. So if you’d like to fork the repo to, for example:

  • provide scanning
  • use native TTS on android (or any other format)
  • retarget AzuleJoe to a different pageset
  • import from (hard) or export to (medium) the openboardformat
  • precompute utterances

Then we’d love to see what you can do. I’d also love to hear from people who are used to working with the Microsoft Office file formats.

 

 

 

 

 

 

 

 

Using AzuleJoe and CommuniKate 20 online

So in November we launched CommuniKate 20.  We were, and are, pretty proud of it.

Screen Shot 2014-10-02 at 12.48.54

In the last two months, something a little strange has happened, it’s been very popular, and interestingly we’re seeing more and more traffic to the demo page that you can see here.  We’d intended that the demo page be used by SLTs  to have a look at the pageset before installing it on a device.  However, it turns out that there are  people using it as a first-order communication device:  they are  loading up the very basic CommuniKate demo without any customisation and using it to speak.

Obviously this is a surprise, and a pleasant one – it’s caught be slightly wrong footed thought and too a certain extent I’m scrambling to keep up.  This last week I put together a ‘how to use the demo’ video:

It’s a very short video – I wanted to show off the functionality without the people watching needing to be literate, and to work with short attention spans. It helps that the software is pretty low in functionality just yet…

Presenting CommuniKate 20!

Screen Shot 2014-10-02 at 12.48.54

CommuniKate 20 on an iPad via the online demo.

Nine months ago I started working with Kate McCallum on a project that was pretty important to both of us. You will remember Kate from showing us how to hack a switch and from giving us a guide to creating a tactile overlay.

Some years ago Kate built a page-set called CommuniKate for her little brother. Fifteen years of AAC experience later this page-set has been used by people with a wide range of needs and on a diverse array of devices.

In January this year we found some time to compare notes at BETT. We’d both been convinced for some time that the AAC world needed open-access transferable page sets that could be used by any user on any device. I’d been researching the issue since 2010 and Kate is a veteran had years of supporting users who had been switched from one page-set to another.

 

CommuniKate 20 running on Grid Player

CommuniKate 20 running on Grid Player.

We decided that CommuniKate should be the first open-access transferable page set. Nine months of hard slog later, it is.

Releasing a page-set is a lot of work. There were a few different layouts of CommuniKate that Kate had to draw from to create the ‘master’ version. Every icon had to be redone using creative-commons images so that we could release the whole set under an open licence. The manual had to be written. A demo had to be coded up. We had to port the master version of CommuniKate 20 over to our launch systems.

Because we’re releasing it openly people can download, use, modify and share CommuniKate and the manual without having to worry about permissions. And because it’s clear we where not going to make any money out of it we found we had a lot of help. Will Wade of the ACE centre stepped in for two of the translations, Paulo Ricca coordinated the team that gave us a Portuguese translation. The guys at DART have just started a Swedish one. SLTs stepped up to give us feedback on structure, nerds stepped up to give us feedback on code. Proofreaders, family, and friends pulled together for us on this one and we are incredibly grateful. We received material contributions from the guys at AssistiveWare and at CoughDrop, who recognised that this isn’t about competition – it’s about making the whole area stronger.

The CommuniKate homepage is at this link. We’d like you to explore the demo, have a glance through the manual and generally look around. If people have feedback, please don’t hesitate, if people would be willing to help translate even five slides to another language or please don’t hesitate. And if you are willing to transfer CommuniKate to a device not on this list, then definitely don’t hesitate.