Layers of Autonomy

Last year I gave an interview to the New York Times magazine cover story on autonomous driving cars that hit too close to home:

‘‘If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore — well, now your car just sees a stop sign. The chances of all that happening are diminishingly small — it’s very, very unlikely — but the problem is we will have millions of these cars. The very unlikely will happen all the time.’’

Unfortunately, the recent fatal Tesla accident which has been thoroughly reported involved the side of a truck trailer. In this case what we know, so far, is that the side of the trailer was white and thin, and that the sky was bright, possibly washing out the camera with glare or blooming from overexposure. There are so many responses in the blogosphere already. The techno-optimists say that Tesla will tweak their code so this particular case does not happen again. Special case after special case which, any programmer will tell you, has a logistical tale of accidental side effects that only increase as the baggage of special cases dragged along blows up in the programmer’s face. Statistician/demographers have already explained that these rare cases are acceptable, because the average risk of death from car accidents still goes down as cars automate. But of course this begs the question, just how are we measuring the mean? Am I mean? If I pay attention when I drive, never text, and don’t drink, then which is safer, me or an autonomous car? Fundamentally, we can make the world safer whilst creating a lottery system for accidents. How does this redistribute error and harm in society, and when is this ethical and unethical? There is much, much more to this than statistics or bug-tweaking. There are underlying questions about interaction design: do we design autonomy to replace people in such ways that new forms of error surface, or do we empower people to become incrementally safer, even if it means our technological trajectory is slower and more intentional? You know where I stand.

AI for Empathy?

Mark Blunden of the Evening Standard yesterday, London, wrote about Amelia, an AI now replacing human council workers in Enfield, England. Two quotes are especially telling; from James Rolfe of council government there:

The customer shouldn’t see that they are interacting with a digital agent, it should be a seamless experience.

And from the president of the company selling the robot, IPSoft:

[not about AI] replacing labour with cheaper labour, but replacing labour with cognitive systems – to be able to answerr a question as a human would understand it.

So we are saving money (not counting externalities of course), replacing human-facing high-touch verbal service with AI, and we are doing it whilst trying to ensure customers will not even realize they are speaking with a non-human entity. What is the formula here? AI – Humanity = Cost Savings – Empathy?

Here is the original article as printed:



Technology in Education

Sarfraz Manzoor writes an article on Technology in Education well worth reading in The Guardian.  Steiner schools eschew the use of tablets and computers in the classroom, and of course this turns out not to be the handicap many assume it will be. In fact many readers will be even more surprised to see a number of studies noting that lessened use of technology in classrooms often correlates with more learning. Andreas Schleicher is quoted- I saw his keynote at the Global Education & Skill Forum- talking about how their OECD international analyses have not yet shown technology to improve learning. This ought to be a wake-up call that we cannot simply invent new technology and throw it at education; we need to be more ethnographic, more aware of just what good learning means, and how to co-design changes to the learning environment with the very important stakeholders we keep ignoring: teachers, students, parents.  Good article, Manzoor.


Surveillance trumps passwords?

Always nice to find a use of the word trump with a small t.  In The Guardian, Alex Hern writes a short article from Google I/O about Android’s intention to reduce the use of passwords by combining multifactorial user behavior evidence together: how you type and swipe, what you look like, where you are, how you sound et cetera– to create  “Trust API” that determines how likely it is that you are, in fact, the device’s owner.  The semantic inflation of calling this “Trust” is already noteworthy– another very human, sociological term turned on its head through techno-semantic inflation. But it is also worth noting that the trope is trust, and the means is always-on surveillance: the ability of interactive devices around us to measure every aspect of our behavior. Always.


Layers of Technology

In this month’s The Atlantic, Matthew Shaer pens an important article, A Reasonable Doubt, about the technology and sociology of DNA matching and its use in jurisprudence. One of the recurring themes we often consider vis a vis Robot Futures is how power relationships are influenced by technological progress, and there are multiple layers to consider in the case of DNA testing, all borne out by this article. First, there is the initial presumption of infallibility many of us have because of our pro-technology bias. Of course we discover that the black and white is in fact grey (thanks Stephen Crane) and that the technology of DNA matching, given a mixed sample, is so incredibly qualitative that ten labs can provide ten answers. Depressing, but eye-opening. But then there is the forward technology march: let’s solve the problem by removing humans from the equation! Shaer goes on to describe a Pittsburgh startup that uses a trade-secret protected algorithm to match DNA samples automatically, without depending on human judgement. Much as war-fighting robots are supposed to avoid human ethical shortcoming by avoiding human decision-making, so this DNA matcher is seen in a dozen states as the new gold standard because there are fewer places for a lab worker to be directly involved. But, ironically, the algorithm is hidden behind the veil of a trade secret, and so our ability to truly audit the process goes missing, and our faith in technology is only further amped. We need a new field, a Sociology of Technology field, similar to VTSS but more short-term; we need to understand just how we identify shortcomings in technology-human systems, and then how our techno-optimist solutions often drive the invention of yet newer problems. Cognitive Tutors, AI assistants, Automated DNA matching– all of them are so much more nuanced in systems-level effects that we make them out to be.

Community Empowerment works

The last chapter of Robot Futures talks about concept of technology as a tool for community empowerment; at the CREATE Lab we developed one specific example of such a system for a Coke plant near Pittsburgh, and the Allegheny County Clean Air Now newsletter has a wonderfully detailed article talking about the positive social impact of such empowerment technology in a very real case.