Wednesday 22 April 2015

Somability - Interactive gesture technology on the Raspberry Pi.



The Gesture Based Technology Professional Learning Community (PLC) owes it's existence to Somantics- an interactive gesture based iPad, Kinect and camera program developed by Cariad at Cardifff Met University in 2011.
www.somantics.org

It is free, designed for purpose and gets great results and engagement from otherwise very hard to engage pupils- it encourages failsafe and creative interactions using technology as an intermediary to activities and interaction.

We had such great results from Somantics, as did the other schools that began using it then on the PLC, that we expanded and sourced a whole load of other Kinect based interactive programs- all of which can be found on the wiki:
https://kinectsen.wikispaces.com/
and in the blog post I did on our Kinect Interactive Room:
http://gesturetechsen.blogspot.co.uk/2014/10/the-movement-room-kinect-set-up-for.html

Well now the team are back with Somability- developed with Rhondda Cynon Taf Skills for Independence and coded alongside it's users in a series of workshops.

It's already out as a free download for the Xbox Kinect sensor from here:
http://cariadinteractive.com/2015/03/26/somability-ready-for-release/

Hopefully it'll be out soon to download on the Microsoft Store for KinectV2, can't wait for that.

But that's not the only news!

It'll be on the Raspberry Pi as well!  I've spent the last two days at the Fab Lab at Cardiff Met University setting up the Raspberry Pi and taking a look at the coding process using Openframeworks and C++.
Why is this good news- well, you won't need an expensive PC, laptop or an iPad to run Somability- you can buy a Raspberry Pi kit (for under £100) and use it on that.  So that's a massive leap.

This is the home screen for the Pi version of Somability:

There's the sound responsive (and programmable) Sound Wheel (make a noise and the shapes move around then the picture changes):

Flow- where the lights respond and follow camera movement:

Space- where movement makes the coloured shapes go all funny:

There's others too- like the mirror but I didn't get photos of that- that's the Raspberry camera on the keyboard with the little red light on it- costs about £10.

It's not out just yet but when it does it'll hopefully be availiable as a kit- so you can buy all the bits you need at the same time- and then of course you also have a Raspberry Pi kit for coding, Minecraft and everything else you can do with that wonderful bit of kit.
https://www.raspberrypi.org/

Good times!

Anthony Rhys
Trinity Fields School
Naace SEN Impact Award Winner 2015

Tuesday 21 April 2015

Meeting Pupils' Needs with New Technology?

Do we fit the technology around the pupils or the pupils around the technology?

I'll be honest- I don't really get this question, it's a bit 'chicken and the eggish' (I don't get that question either).  Maybe the best questions are the ones that don't have easy answers.

I guess it means, do you have a pupil then fit the technology you use around that pupil's needs- or do you get technology and then think 'who can we use this with?'

Ideally you'd think of the pupil's needs first then fit the technology around them I suppose- to help them with their independence, communication, socialisation and engagement.

But- what about when something new comes out?  Then you have to think 'who can we use this with?' - there's no way round that, and you trial and try things and you fail and you go back again to some other pupils and if it helps you keep it, if it fails you don't use it.

But also even if you are experienced in fitting the technology around the pupil needs- say for instance you have used an iPad app with Pupil A to achieve one thing- who's to say using the same app with Pupil B for something completely different won't work really well?

More often than not you'll use your 'experience' of success with Pupil A to try and replicate that success with Pupil C- and, possibly (probably!) it won't work.

In my experience using new technology with pupils is a constant state of trying things, experimenting, failing, thinking, re-thinking and reflecting on what you are hoping to achieve- and also always keeping in the front of your mind- am I using this technology just because I can or because it's actually helping?

A few times I have been asked a question like 'What's a good iPad app?' or 'What's the best eyegaze program?'

That's like asking 'What's the best book?'  I don't know- it depends on what they like and what you want them to achieve- try some different things.

Will a Kinect program work with your class? will an Eyegaze program work with Pupil A, B or C? I've no idea.

The best I can offer is that there is a broad consensus that technology is beneficial to pupils with complex educational needs and that it does enable them to do things they otherwise are unable to do.

The leg work is up to you as an educator, in the same way you take time to find out what music the pupil likes, what peers and staff they like, what textures, tastes, sounds and smells they like.

Here's an example of the complexity:

The 7 pupils in my class are all on a P3/P4 level, all with no functional verbal language and complex physical and learning disabilities.  They are a 'cohesive' group in that way (when compared with other pupils in school) but that cohesiveness is extremely arbitrary.  They all have widely different IEP targets reflecting their widely different personal needs.

After 3 years of teaching them this is the technology they use in class:

Pupil 1: This pupil has very limited movement but is a great switch user. Uses eyegaze with only 'tinkly' musical programs or disengages. iPad on head mount using Airharp camera app or, when really calm, a Zither touch app. Two jellybean switches- both head mounted on bendy arms to operate a switch operated stereo, switch operated sequenced big mack, food blender and lights.

Pupil 2: This pupil has some limited movement but well motivated by technology.  Uses eyegaze- light and noise based particle programs, some cause and effect apps.  Head mounted switch with stereo, two head mounted switches with disco lights and music, iPad with light and noise based touch apps, active and passive bright light apps, Oculus Rift with bright effect programs, whiteboard and switch with some song programs, especially with staff. Interactive floor projector with particle programs.

Pupil 3: This pupil has some good mobility but is very unmotivated by most things, prefers tactile items to mouth, mirrors or people.  Uses touch whiteboard with particle programs, iPad (in very tough case) with Electra, Gravitarium or firework apps. Interactive floor projector with particle programs.

Pupil 4: This pupil has very limited movement and very high medical needs. Uses switch operated stereo, 'daft' iPad apps with 1-1 staff support.

Pupil 5: This pupil has good movement but incredibly hard to engage, prefers to flap items. Uses iPad fireworks app, switch operated lights in dark room. Leap motion with musical program, Beamz.

Pupil 6: This pupil has good movement but very hard to engage, prefers to watch people and mouth objects.  Uses iPad touch music apps with vibrating speaker. switch operated lights in dark room. Interactive floor projector with particle and splat programs.

Pupil 7: This pupil is extremely tactile defensive, has VI and is very hard to engage.  Uses iPad speech repeaters, Looper Dooper sound repeater, songs, some touch music apps. Switch operated stereo on occasions.

Some of my class rely heavily on technology, others aren't really that fussed on it and prefer touch, taste and people.  They all use switches but for different things, they all use iPads but again they all like completely different apps, two use eyegaze, three use the floor projector, one the Beamz, one the Leap Motion, one the Oculus.  None of my class use the Kinect, they either can't see it or they just don't respond to it.

I still try new things with them. If they like it they engage with it, if they don't like it they don't engage, or they throw it away, or they cry or hit out.  When they do engage we keep developing and nurturing that engagement.

It's trial and error and teaching in an experimental 'give it a go' environment - one where you keep trying and never 'finish', where you embrace and develop what works and drop things that don't (after you've tried them a few times- just in case....)

I think success in using technology with pupils with complex educational needs lies primarily on:

How much you already know about your pupils- what they like, where they are and what they can achieve.
The variety of what technology you have available in your 'Let's give this a go..' toolbag.
and how often you are willing to try (and fail).

Anthony Rhys
Trinity Fields School






Wednesday 8 October 2014

'The Movement Room' a Kinect set up for pupils with Severe Learning Difficulties

This post will describe the 'Movement Room' set up in our ICT suite and how we get the most out of our Kinect in our special school.

The interactive technology in the room has been two years in the making- with help from all around the world from California to Cardiff and from Sydney to Jakarta.

Firstly- the computers at the back of the room.
There's a iMac plugged into one Kinect which runs the Processing sketches, Somantics, Noise Ink and Z-Vector, this is plugged into the smartboard.
Then there's a PC plugged into another Kinect which runs Visikord.


The Kinect extension wires travel across the room under a masking tape cover for health and safety reasons......



On the other side then is the Smart wall projector- projecting onto a blank white wall.  The two kinects are there too- one on a table and one on a stand (two because they are attached one to each computer). Oooh look, there's me waving too.
The projector is connected to both computers- so switching between input1 and HDMI enables you to swap screens in seconds between the iMac and the PC.



We also use choice boards for pupils to choose what programs they want to interact with- either the Somantics main screen print out or a home made board of sketch photos. 


So the pupils can choose from the ten Somantics apps, Visikord, Noise Ink, Follow, Kinect Physics, Lines Around the Body, Kinect Flow, Missouri Paint, Particles, Evil Twin and Rainbow Man.

Which means when the lights are off you've got the best Kinect-based interactive room in the world (possibly) and with some music on you've got the best Kinect-Disco room in the school!


For more information on the set up and the programs used please see the PLC's main Kinect information pages portal link here:

http://kinectsen.wikispaces.com/  The site will tell you all you need to know about Kinect and SEN in schools.

This video includes clips of the room in all it's incarnations over the last two years. 




Thanks to Wendy in Cardiff for Somantics, Davor in California for Visikord, Trent Brooks in Sydney for Noise Ink, Chris Riebschlager in Missouri for Kinect Art, Adityo Pratomo in Jakarta for Evil Twin and Rainbow Man, Amnon Owed in The Hague for Kinect Physics and Flow, all the other Processing Programmers out there and our very own Matt from Caerphilly for setting all the technology up.  

This room and the priceless interactions within would not have been possible without all of you. 

Anthony Rhys
Trinity Fields School
Ystrad Mynach

Sunday 14 September 2014

Virtual Reality and Special Needs- or 'Is it ethical to send pupils into space?'

During our PLC meeting on the 16th October we finally got the Oculus Rift working thanks to James Winchester and his HDMI enabled laptop!

What is VR?  

Virtual Reality- using technology to create an immersive world/ experience. 

What is the Oculus Rift?  

Arguably the latest in this line of technology- it's just come out as a Development2 kit- which means it is not 'available in the shops' and there's not much commercial software for it yet- it's for programmers to try out and make content for.  But- it is very close to the finished product and there are many examples of programs out already.

Basically it is a headset plugged into your computer that you wear over your eyes.  Inside are two screens- one for each eye and these combine to give the impression that you can 'see' another world- a 'Virtual Reality'.  Another camera attached to the computer then watches how your head is moving so you can look around this world and it moves accordingly- you can look around objects, under tables and also 'walk' forward- usually by tilting the head or using the keyboard or a controller. 

Here is the headset and the camera attachments.



And this is what you see on the screen-  two pictures that when viewed through the Oculus Rift create the Virtual Reality effect. 

Before I move on I need to say that a great teacher in Australia, Mathieu Marunczyn, has already been trying the Oculus Rift with his pupils at a special school and he has written many interesting pieces on it's potential use in education and special needs- so we are following on from this and hopefully adding to his work. 

Oculus:

Special Ed and Oculus

A lovely case study of Oculus and an ASD pupil.

Mat's blog is also great for a host of other things too- it's a shame he is on the other side of the world from us but we do keep in touch through Twitter- and it's great to have others trying out technology in such an inventive way.

What can it be used for?

I see three main aspects in our schools- though they're not totally separate from each other too. 

Creating a learning environment.

For example- learning about space and the planets?  Then fly around space and the planets.  
Learning about a period in history?  Then walk around a virtual Coliseum or Iron Age village or Stonehenge- what better way to spark discussion and thought than by sort of 'being there'. 
Creative writing?  Then go underwater or into a spooky house and use it as a platform. 

The possibilities here, especially in mainstream or MLD classes are legion really if you think about it. 

Also there is the idea to use VR as a 'role-play' space for learning key skills.  Here is Mat's initial thoughts on the issue. 


I can see how could work as an element towards such things as travel training and social skills work.

Use as 'calming' environment for pupils. 

Mat has spoken about the possibility of pupils using it to calm down and manage their own sensory processing problems.  With headphones on the experience is very immersive in terms of vision and sound and Mat gives the report of a pupil using it for the first time here with the Blue Marble program.


If, for example, due to an unavoidable classroom set up a pupil can use VR as a relaxation technique to regulate and change their sensory input then that could be a valuable tool. 

To enable pupils to have new experiences.

This is linked to the first one I suppose but I'm considering here pupils with severe physical needs and also severe learning disabilities (SLD).  

If during a sensory story on the sea a pupil could experience an immersive visual and audio experience of being on or under the sea wouldn't that be great?  

Or during a lesson themed on the Funfair a pupil could 'experience' a rollercoaster ride for themselves. Or fly around space during your Star Wars sensory story? Or travel up a mountain?

Surely some of these experiences would be very difficult to emulate in the real world- so why not let them have that experience themselves?


What are the implications/ considerations for our pupils?

It's not the 'real world' so any social or life skills learning would have to be backed up by real world examples.  

I don't see much of an issue with this as already if we teach a skill we usually start in class to practice and teach some key concepts- then it is taken into the real situation and taught there too. VR would just be another tool in this and a very useful one. 

Do we want some of our pupils to have another world to immerse themselves in?  We spend a lot of our teaching time with some pupils getting them to interact with others and respond to the world around them- do we want to add another layer- another world to complicate things?  I think this is something to consider with pupils - as it is with anyone who spends too much time in the 'gaming' world. 

Is it ethical to send pupils into space?  I wouldn't feel comfortable strapping this onto a pupil's head and sending them down rollercoasters without their consent- and with some pupils how would we get this consent?  This would link in directly to general ethical issues on how we treat our pupils in school- keeping them informed and treating them with respect.  

Conclusion

There is no conclusion.  Hopefully as a group we can discuss some of these issues and come to the beginnings of an informed consensus on the issues that the emergence of VR brings to education and the world of special needs. 






Thursday 11 September 2014

Professional Learning Community Overview

Welcome to the new Gesture Based Technology SEN PLC.


Link here for the Home site for the Gesture Based Technology PLC.   The site deals with using new technologies for pupils with PMLD/SLD/CLDD.

Our first meeting as a new group is on Thursday October 16th at Trinity Fields School, Ystrad Mynach. (9am for 9:30 start.  Finish 3pm)  All welcome- but let us know first!

Our central aim is how we use and analyse video evidence as in our work we record a lot of video but what do we do with it afterwards?  We will be looking at Video Analysis techniques that are currently used in sports but applying them to educational situations.

Work has already begun on this at Exeter House School on analysing Intensive Interaction sessions- see Andrew Walkers work with AnalysisPro here.

The PLC is co-headed by Exeter House School in Salisbury and Trinity Fields School in Caerphilly but all members are equal and can contribute as much or as little to the group as they can.

There will be a requirement to agree on how we will analyse video evidence and things out in class.

We will also be looking at new developments in Gesture Based Technology throughout the life of the PLC with practical and evidence demonstrations- this will include Kinect, Eyegaze, Leap Motion, Oculus Rift, iPads and touchscreens.

What is Gesture Based Technology?  How is it relevant for pupils with special educational needs?

This video will give you an overview.