There's an archetypal narrative of the web that starts with the word disintermediation. Which is the posh way of saying cutting out the supply chain middle men.

It's usually accompanied by a picture showing before:

and after:

The perenial poster child for the promise of disintermediation is Dell; cutting out the distributors and retailers to go direct to consumers with their marvelous black boxes of technology.

Like most other things in the realm of new technology people tend to take this picture and extrapolate. So many industries have so many middle men it's tempting to imagine a world without. In the creative / spectacular industries in particular there was a promise from the early years of blogging that content producers could go directly to consumers and cut out countless layers of intermediation en route. Music artists wouldn't need record labels, authors wouldn't need publishers, journalists wouldn't need newspapers. And all this would play out on an open web tied together with links, search and micropayments.

But it hasn't really played out that way. Since the last dot com boom and bust the narrative has changed from disintermediation to reintermediation. And from there to the consolidation of reintermediators.

Apple reintermediated the music industry, Sky reintermediated the TV industry, Netflix threatens to reintermediate the cable TV industry, Etsy reintermediated the local craft market, Ebay reintermediated the jumble sale and Facebook reintermediated our friendships.

And for everything else there's Amazon...

...who've managed to reintermediate everything from the book industry to large parts of the film industry to the small power tools industry and onwards.

The weird thing about Amazon is they're always there but rarely noticed. The rest of the tech community is happy to stand on any platform available to shout their own praises. And the tech press follow along like adoring puppies. Every time Ed or Ev or whatever his name is announces a new feature the web lights up. Usually with adoration. It only takes an announcement of an announcement by Zuckerberg to trigger a torrent of praise / condemnation. And every time Google make a public proclamation... well at least Jeff Jarvis gets excited. Meanwhile Amazon just get their heads down and build the best online store and the best engineering platform and the best integrated service design and there's barely a murmur. It's all vaguely weird...

Something else that's weird. When the industry gets together for a spot of communal backslapping the integrated service design prize always heads towards Apple. Which makes me wonder if the people handing out these prizes have ever tried to use iTunes? In general it's pretty horrible. But compared to Amazon's Whispernet / Whispersync and one click purchase it's a real turd of a system. In the time it takes to boot up iTunes you can grab a Kindle, search for a book, find a book, buy a book and start reading it. When it comes to integration of web storage, web services, software services and physical devices Amazon make Apple look like amateurs. And the rest of us are barely trying.

That said Amazon aren't exactly shy about their role as a reintermediator. The self-publishing upload form cuts out the agent, the publisher, the distributor, the wholesaler and the retailer in one swoop. The Author Central and 'ask the author' features plug readers directly into authors, again cutting out the middle men.

But back to the point...

So why the new middle men?

Or why didn't the promise of producer to consumer work out? Wikipedia says:

Reintermediation occurred due to many new problems associated with the e-commerce disintermediation concept, largely centered on the issues associated with the direct-to-consumers model. The high cost of shipping many small orders, massive customer service issues, and confronting the wrath of disintermediated retailers and supply channel partners all presented real obstacles. Huge resources are required to accommodate presales and postsales issues of individual consumers. Before disintermediation, supply chain middlemen acted as salespeople for the producers. Without them, the producer itself would have to handle procuring those customers. Selling online has its own associated costs: developing quality websites, maintaining product information, and marketing expenses all add up. Finally, limiting a product's availability to Internet channels forces the producer to compete with the rest of the Internet for customers' attention, a space that is becoming increasingly crowded over time.

Which is probably true but I suspect it's more than that. Recently there's been a flurry of blog posts all saying that some random industry is "becoming software". It all kicked off when The Wall Street Journal published:


Why is software eating the world

There's a premise that what sets successful businesses apart is their readiness to adapt to a software driven world. I'm not so sure. I've been around enough software for long enough to think most of it is just a patchwork of bug fixes for obscure corner cases and a set of features that no-one can quite remember requesting. Actually, that's not quite true. Software is what we write to extract information from data. The worse your data model is, the more software you have to write. The optimum line count for code is zero. But anyway, software without data is about as much use as a pub without beer. And I'm firmly of the opinion that:

Applications Are Like Fish and Data is Like Wine

Re-read Why software is eating the world and substitute every occurance of the word software for the word data. I swear it makes more sense.

The attention graph

In my head there's a picture that looks something like:


Our friends in social media world tend to worry most about:

Enterprise architects, IAs, archivists, taxonomists etc tend to worry most about:


But the one thing all the usual web whatever-number-we're-up-to suspects really get right is:


Facebook's creepy Open Graph protocol and "frictionless sharing" are just an attempt to own the attention graph no matter where its users are paying attention. The read / write web exists; you read something, Facebook write it to their database.

In my day job I've heard this described as "having a personalisation strategy". Which completely misses the point. Personalisation is the bait, customer relationship is the trap.

Anyway, Amazon take the exploitation of attention data to new levels. Their social graph is minimal and their content graph barely exists. Browse Amazon.wherever-you-are and the majority of the content is contributed by customers. And the majority of the context / navigation is contributed by customers. Web services inviting contributions by users tend to have standard boiler plate terms and conditions that at least pay lip service to the contributors rights over their material:

You or the owner of the content still own the copyright in the content sent to us, but by submitting content to us, you are granting us an unconditional, irrevocable, non-exclusive, royalty-free, fully transferable, perpetual worldwide licence to use, publish or transmit, or to authorise third-parties to use, publish or transmit your content in any format and on any platform, either now known or hereinafter invented.

But Amazon don't even bother with the lip service:

If you do post content or submit material, and unless we indicate otherwise, you (a) grant Amazon.co.uk and its affiliates a non-exclusive, royalty-free and fully sublicensable rights to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, and display such content throughout the world in any media; and (b) Amazon.co.uk and its affiliates and sublicensees the right to use the name that you submit in connection with such content, if they choose. You agree that the rights you grant above are irrevocable during the entire period of protection of your intellectual property rights associated with such content and material. You agree to waive your right to be identified as the author of such content and your right to object to derogatory treatment of such content. You agree to perform all further acts necessary to perfect any of the above rights granted by you to Amazon.co.uk, including the execution of deeds and documents, at the request of Amazon.co.uk.

What you contribute to Amazon belongs to Amazon.

Pervasive computing, ubiquitous surveillance

By now we've probably all sat through conference talks on the "internet of things" and pervasive / ubiquitous computing. Past the point where people stop talking about making rabbits twitch their ears when someone tweets about carrots, I think the Kindle is the first real world example of any of this. So it's still got a screen but it doesn't feel like a computer. It's a reading device, a retail terminal and a beautifully designed back channel.

Read the Whispersync marketing foo-foo and the public messaging is all about seamless synching between devices. So you put down your Kindle and open the Kindle app on your iPad and your book is miraculously open at the page you were reading. Obviously there's no device to device synching going on here. It's device to web service to device. So all that data gets phoned home to Amazon.

Amazon already know the books you've bought, the books you've browsed, the books that other people who've bought similar books to you have bought, the books you've rated, the books you've listed, the books you've reviewed. Now if you're using a Kindle with Whispersync they also know if you're a slow reader. They know if you're the kind of person who buys books and never makes it to the end. They know the books you've bought that you skim through in an afternoon. They know the books you read slowly, flicking back though pages to check facts.

Again, this kind of integrated end-to-end service design is interesting to compare with the supposed masters of this sort of stuff. Apple managed to build the iTunes store, some clumsy iTunes software and the rather lovely iPod. But they allowed the backchannel data to get intermediated by Last.fm / Audioscrobbler. You just can't imagine Amazon letting Good Reads or Open Bookmarks build a business by tapping into the Kindle backchannel. They just seem to understand that connected technology isn't just good for distributing content outwards; it's also rather well adapted to reporting back on the usage of that content.

I'm probably gonna get flamed if I've got my facts wrong here but I've searched long and hard for Kindle / Whispersync terms and conditions about user contributed content / data and they just don't seem to exist. So I'm assuming that Amazon terms and conditions cover Kindle Whispersync too. In which case all your Kindle reading data, your bookmarks and your margin notes belong to Amazon too.

I can't help but wonder what it would be like to hack with that kind of data. What could you build around community reading groups, formal education, adult literacy? At the very least it would save me the chore of ticking homework diaries. But I doubt we'll get that chance.

Facebook regularly take a beating for pushing privacy issues to breaking point. I'm not sure what the difference is between Facebook snooping on your reading and Amazon snooping on your reading except the commonly reported privacy issues around Facebook are all about what gets reflected back onto the web. As opposed to what gets absorbed by Amazon.

The usual answer to all of these privacy worries is, what's the worst that could happen? I get some better adverts to watch. Which fits firmly in the, "nothing to hide, nothing to fear" bucket. And it's equally nonsense. It's not like there's not past evidence of data being collected for perfectly innocent purposes being used for something altogether different.

The obligatory user experience bit

Since most of this blog is about user experience in one form or another I'm feeling duty bound to pitch in with: forget about HTML5 and CSS3 and jQuery and responsive design and "mobile first". The most important  issue for user experience people to grapple with is informed consent. More and more web services are dependent on user contributed content and data. Every time you make a contribution (explicit or implicit) you're trading convenience for privacy. This isn't necessarily a bad thing; it's something we do everyday in real life from mobile phones to loyalty cards. But as the web moves out of the browser and into smart objects, the trade-offs we're making need to be made explicit so people can make informed choices about when to get involved and when to back away.

There's another personal bugbear in all of this. We're increasing told that companies fail when "UX professionals" don't have a C*O seat at the top table. The company would make better products that more people would want to own and use; everybody wins. Again the poster child is Apple and the pin up boy Jonathan Ive. But in the world of constantly connected, reporting devices your employer's interests are not necessarily the same as those of your users. There's a more general question here: as the supposed representatives of "the user" who exactly are we working for?

A dystopian summary

It feels like we've long since lost the desire for an open, disintermediating web. The old power brokers of publishers, record companies and film studios are dead or dying. But we're all more than willing to trade our privacy for the convenience offered by the new intermediators.

It's not like you can just ignore what's going on. Owning your own domain isn't enough anymore. If you're a newspaper and you don't engage on Facebook you'll just miss a massive chunk of your audience. If you're a broadcaster and you don't promote your programmes on YouTube you'll miss a huge chunk of your audience. If you're a publisher and you don't go through Amazon you might as well give up and go home.

But all the while you're being cut out of the deal. Signing up to any of the new intermediators isn't about opening up new distribution channels; it's about outsourcing your customer relationship. And attempting to fight the reintermediators on your own turf only works if you imagine that customers would get more value out a relationship with Penguin than they would out of a relationship with Amazon.

I'm not sure what happens next. Maybe the web is just going through a period of reintermediation and consolidation. More optimistic voices than mine would point to things like Disapora, data portability and personal data stores and say the web will return to a more open, distributed model. But for all the talk of the web as a distributed system it kinda isn't. It's some servers and some clients. Querying distributed data sources to build the kind of "experiences" that people have got used to isn't close to happening. Even in the occasionally rarified world of Linked Data, reintermediation is happening in the form of consolidated data stores / market places like Kasabi. Because having all that data out there isn't enough. It has to be in one place to make it useful to query.

I have nothing but admiration for Amazon. They make the best shop. They make the best services. They make the best software. They have the best data. But they scare me. If I was a book publisher they'd terrify me. And if I was a film studio or broadcaster I'd be watching my back. Especially since they also own LoveFilm and IMDB. And especially when they've just announced Whispersync for movies and TV. And even if I was Google or Facebook or Apple I'd be concerned. Because Amazon are light years ahead of the game we think we're playing.

Right now the bits of the web that aren't Amazon or Google or Apple or Facebook feel like the high street greengrocer waiting for the supermarket to ride into town. Which depresses me because it's the exact opposite of what got me interested in the web in the first place. Maybe there's still room for a chi-chi artisan cheese shop or two. But who the hell wants to run an artisan cheese shop?