I Have A Dream: My Personal API is almost here! (Part 2)

(Almost) as soon as I hit the "Submit" button on my last post, I see this post from RWW on a new startup (but really built on online storage company Mozy and an acquired personal information aggregator startup, into a new service called Decho. This makes my dream a lot closer, because it provides the backbone infrastructure for app developers (of which I am one) to build on top of this infrastructure and bring the dream closer. The key factor here, as one commenter posted is the UI, but I also think that user-driven control of their own data is key: it must be really user-driven, very granular (share my gizmo interests/profile with Jack, but not with Jill, tc. and ONLY when I am in a browsing-gizmos sites mode, versus a business mode. And while you're at it, also don't share that data with Walmart for Black Friday, but only with my local gizmo retailer company - I had to give an extreme example, lol). Now Decho is not quite there because it takes a lot more than user-owned digital data, e.g. online behavioral data. The crux is to build an ecosystem that convinces the BIG Data Owners (those that have a LOT of your data but that don't necessarily use it for anything else than to enrich/optimize on their own core business proposition) that there is more value in data itself beyond using it for operational purposes.

For instance: the Walmarts of the world own a TON of data, BUT: they're not optimizing it per se (other than building CRM systems to improve their own sales goals via marketing, promotions, etc.). If/when they will realize that just owning that data (but then sharing its control with the consumer) - thus extending their core value prop to a more platform-y, data network secondary business, that will bring the dream closer.

But of course, this must start somewhere and Decho seems to be on the right track here. Thoughts? More to come tomorrow.

I Have A Dream: Where is my personal API? (Part 1)

The words bubbling in the mouths of every tech, (digital) marketing person nowadays are: API, Web 2.0, social web, social networks, etc. etc. And on, and on on it goes. And frankly I am sick of it! Why? Because with a few exceptions, nobody figured it out yet; no wonder Facebook and other social and 2.0 companies are not yet making money. At the Web 3.0 Conference & Expo a couple of months ago in Santa Clara, I wanted to provide an open forum to exploring the new business models of the future of the web. The conference was a success in my opinion, and we are now planning for the next instantiation, stronger, better, bigger, more encompassing, in New York in the Spring (date tentatively set). But as it turned out, a lot of the focus of presenting companies was (still) around advertising and targeting as moentization approach to the next web. The truth is, I really believe that's only one, and perhaps not even the most important one. Let me explain.

Certainly advertising has potential as monetization tactic of 3.0, but that's only because it's "tried and true", and it's the way we have been used to sell VC's in the Internet startups we are dreaming. What I wanted to achieve (and will do so even more strongly at the next event) is to force people to think other ways of monetizing the next Internet. Since it's all about data, we tend to reduce the conversations around using data (on the backend) to come up with smarter, more relevant ways to target and advertise to people. That may be fine, but I really am dreaming of monetizing data itself, not just the ads that may come out of processing this data.

This is a little more complicated than advertising, and perhaps closer to the business models of data services companies such as Acxiom, Experian, etc. (albeit they are doing it behind the backs of consumers). What I want is what I call the "Dan Grigorovici API". I want to own my data and I want someone to intermediate publishing my own personal API (a personal Mashery if you will, or maybe GoDaddy can add a premium service like that - here, I am giving away ideas for free). With this, I then want to allow only certain services (other my friends' APIs, maybe Walmart's API only when they have a sale for an item I wanted for so long, but only if it's less than $20, etc. etc.), and I want to control it. In other words, I am sick of what i call "the car dealer marketing" (you know what I mean), and I want me to do the dealing. Of course, this may become a little complicated when I have a day job, kids, and other things to take care of, but then (or for consumers who don't feel like being so active), another company/provider can do it all for me  on my behalf. For this type of service, I am willing to share my most private data. Why? Well let's put it this way: instead of wasting my data in millions of places that I have no clue about, I'd rather share with one central hub (who needs my permission to share/sell it to an Acxiom or such), and if/when something goes wrong, I don't need to dig through tons of channels to trace back where my data has moved from a network to another; I can just go to this one provider and scream at one, not at many (most of which are invisible to me).

A few days ago I was talking to a friend of mine, a developer, sharing the same idea and even though we have the technology available to implement this now, the biggest barriers are more on the business side (as usual, actually):

1. Users (with a few exceptions) are unaware of the value of their own data. They have no clue what invaluable an asset this is;

2. Even though we are well on the way to decoupling data from the technologies/environments that helps create them, for the most part, they are still pretty stuck to each other (including Facebook, mind you).

3. (the scariest of them all) It seems like a sisyphean work to change an entire way of doing business (from our pretty standard advertising, no matter how "relevant", to this one I am talking about). I firmly believe that we, as consumers, should start a silent revolution, maybe beginning as the "no TV Day", with a "no data trail day" - a day every year where consumers refuse to give away data to all service providers we are constantly transacting with.

How does this relate to monetization and new business models? It does, because if I am in control of my own API, the balance of power changes from the "car dealer economy" to me and my trusted intermediaries. And then not only I can charge for sharing my data, but I can even pay for these type of value-added services. And in this vision, I probably won't need a smidgeon of ad banners at all, because what's the purpose? This is inherently data-monetizing.

I am really obsessed with this dream I have (and there are a few startups I know, including my own project, plus Project VRM) that have been slowly working on something like this. I will post more in the next days about this, but would like to hear your dreams and what you think about this.

My final point is: no matter how many agencies, advertisers (GM opening a Saturn owners social network, and other crap like that), technopundits, and other sad guys keep mumbling "social web", 2.0, blah blah blah simply because it's fashionable, we are STILL monetizing not data, but pages (hence this is why Facebook is still not making money). I am pretty sure platform companies such as Facebook (and pretty positively Google) know the value of their assets is not in pages they own, but in data users generate on their pages. But I believe the reason why their CPM is so low and are not making money and they're promising a way to monetize the social web in a different way is not because they're slow or stupid. It is simply because it's very hard to turn an entire macro-level business on its head (it's the network effect, stupid, so they're waiting for the first startup to be minimally succesful with this, then they will eat it alive and scale it large), and also because the entire advertiser-agency community it stuck on stupid (especially media planning folks, I gotta say).

So, let's start the silent "no data for free day" revolution and until we get there, here's some food for thought: The Personal Platform, by Doc Searls. This is what I am talking about. But let's rush a little, I want to see the fruits of my dream in my own lifetime. Where's my Dan Grigorovici API?

The expected utility function of open data

I went over an earlier post and realized my message probably was hard to understand because of the poor english in it (apologies, it was late when I posted it, and did it from a stream of consciousness). I would like to re-open this conversation because I believe this is the nut we need to crack to monetize data, and not pages. I will repost the original (but cleaner) posts, with additional explanations of my thinking:

 

There is a fundamental conflict between user needs (e.g., I want all my data to be mine, portable and across all data providers/sources/sites, etc.) and data owners (big Internet companies, sites, networks, Walmarts, Amazon's, your grocery store, your medical insurance, anyone holding, processing, doing something to the data you leave behind in your behavioral trail of interacting with them). The data owners must be convinced (in a granular fashion, by answering every parameter in the "expected utility function" equation) of the necessity for them to open up their data. Some of the parameters in this equation that needs addressed are: size of payout to data owners of opening up their data to developers/users; probability of occurence (of this payout, which is essentially the needed business models of Web 3.0 app providers); risk aversion (check the panel Business Risks of Web3.0: What Risks? at Web 3.0 Conference & Expo); we also need to account for differential utility of the same payout to different companies/verticals with different assets and/or needs.Dan Grigorovici, Web3Beat:, Oct 2008

Now, I don't think we've done a big deal of work on this so what we need to do now is to frame our value prop to Big Data Owners from this perspective, by answering these basic questions. If we can't, we won't get the data because the owners will not feel incentivized to do so. And quite frankly, very few of them do. What I mean when I often say "monetize data, not pages" is really that the new business models of Web 3.0 are not as much about advertising (or at least not the current shape of advertising/marketing): it's more about viewing data as an asset on its own, not so much for ad targeting but rather relationship (true social, consumer-driven relationship) building. It's what relationship marketing was supposed to be, before we started to use theocratic concepts such as "own", "manage", "acquire", "retain" customers, as if we ever had so much power on them (wake up: we never really had, outside of our CRM-ish PPT dreams).

 

Food for thought 1: a cool 3.0 product idea

I was talking to a friend of mine the other day, and he came up with this idea:

"I live on a block at one end of which there is a Duane Reade, and at the other a CVS. There are some items that are on sale/cheaper at one, and others cheaper at the other. Why can't someone build a smart consumer-driven shopping agent that given a shopping list will tell me at any point, which items to buy from which store?"

Kind of like a smart consumer shopping optimization engine, right? I thought the idea was brilliant in that it exposes directly the fact that the success of this is NOT a technology issue, but a business and product marketing issue. Here's why: while I think this is very doable technologically (biz dev with Duane Reade and CVS to access their inventory and/or promotion offer API's - if they exist, if not, create it/scrape the site, etc., etc.), the business value prop for them is painful and hits right at the core of the substantive dichotomy between consumer needs and data owner needs that I mentioned in an earlier post; while I want to have this capability as a consumer, the incentive for the data owner (CVS, Duane Reade) is yet unclear for why they should open up their data. This has to do with the mentality of "opaque" branding/selling/marketing, as opposed to "transparent". There are two issues here, and I would like to hear your opinions/answers on these (please consider i am the Chief Data Officer at CVS):

1. Why would I want to provide this service to the consumer, and open up my offer data? Since I plan my promotional efforts in part based on what Duane Reade offers, to get more in store foot traffic, why would I risk them knowing my every move?

2. How do I make this process more efficient, given that my promotional offers change on a daily basis as function of manufacturer/CPG deals, market demand, competitor (Duane Reade) promotion counter-offers, holiday retail, location, etc., etc.? In other words, how do you see this service being intelligent but also dynamic enough to pull of these data points considering that this process is mostly manual today?

Any takers? By the way, this is the first in a series of "wild and crazy" posts that will actively seek reader comments in solving a concrete idea.

A Big Deal: New York Times opens up their data

A few weeks ago, New York Times has opened up a series of API's (the Campaign Finance, Community, MovieReviews, and TimesTags APIs). This is a very big deal for me, as it shows further proof (as I said several times in previous posts) that having data is not as useful as doing something with it (in terms of processing it). From this respect, I've always viewed New York Times (or any other Big Media companies) and their content as perfect example of Big Data Owners. From this perspective, any company that collects user/consumer behavior data as BYPRODUCT of a direct-to-user value proposition should seriously consider monetizing their data by opening it up, and allowing developers to mesh it with other data, to provide users/consumers with valuable services. Let me explain what I mean by that.

Your local grocery store, Walmart (both your local physical store and walmart.com), your credit card issuer, media content companies (New York Times is an example of this, obviously), your phone company, etc. etc. are examples of companies who built their value proposition and core business model NOT on data, but on retail, consumer credit, content production, communication (in the order I mentioned these companies above). By virtue, and in the process of delivering their market value, they have collected lots of data for purposes of operational efficiency, etc. etc. So their data is NOT their core business but a byproduct. The Web 2.0 world has opened up possibilities for these companies to consider data they own as assets in and of themselves. I seriously think that moving forward, the next generation business and monetization models are data-driven on a virtual network level, in this way:

1. Data owners open up their data (New York Times is a great example of yet another company, aside from Yahoo, etc. etc., that gets it);

2. By virtue of data APIs, developers are able to deliver applications to users that integrate this data with others, in novel and interesting ways;

3. Data owners then monetize this data as asset (in other words, this is not simply monetization=advertising/targeting), but monetizing as knowledge access. E.g.: in the grand scheme of things, no matter how big your site is (whether content or advertiser/brand site), most of the time consumers spend OUTSIDE your site. On the other hand, and because of this, your knowledge of your consumer is substantively limited to the data you own (plus some third party appends). But you will never know how valuable a consumer is (and be able to deliver valuable services to her) by understanding (with her permission) what their interests are OUTSIDE/BEFORE/AFTER they land on your site (which most often is short, very goal-oriented behavior in nature). So, you DO need other people's data.

This means that data should be monetized in and of itself as asset. This is the platformization/network-ization of everything, really. And this is a good thing. New York Times is one of the most recent examples of "old" media/content companies that gets it. I also discussed with various (but not so many) advertisers/data owners from various verticals that have shown interest in this topic, and I think this is great!

But I think, once more, that we as developers and technologists must leverage this growing demand and be able to explain how this works, and why they should do it, and even design the next generation data monetization models. I am interested in your thoughts on this, especially since I have been working on this and think I have a pretty decent idea of how to do it.

Finally, someone that gets it: Yahoo Bigtime Open(ing) + Facebook: read and weep

Yahoo has released Y! OS 1.0 platform! I've tried to play lately with Facebook's API and FQL, for purposes of developing smart applications with non-PII user data, in the spirit of Web 3.0: a big pain in the ass! Facebook's data is not that open at all. It's a silo in disguise. You basically cannot process (therefore store either, because if you are trying to run an algorithmic process on the user data without actually taking the data out, you can't) data in order to, for instance, develop a recommendation system that recommends your friends, or objects of interest between common sets of friends (from school versus work, etc.). This, apparently in the name of user privacy. Truth is that my deep desire as a user is to take my data with me wherever I may be; and mind you, this doesn't mean that I am "stealing" Facebook's traffic, because I can devise my product to send traffic to Facebook. But nooooooo! That just tells you how hypocritical Facebook is: its concept is revolutionary in its understanding of content as object-oriented, but its monetization is soooooo 1.0: pages, pages and more pages (of eyeball traffic). Of course, I believe I have an idea or three about how to monetize Web 3.0 in terms of "monetizing data", not pages, but when I hear of Facebook and others "opening up" their data, I laugh: first off, it's not that open, second of all, so what! They're still monetizing it old school, with R/F eyeballs (with a fairly low CPM if you ask me).

But now, after I heard about it through Y!DN and at the Web 3.0 Conference early this month in Santa Clara, Yahoo seems to have gotten it a lot better, as the TechCrunch article tells you: first off, the application platform allows developers to access your and your friends' activity stream both on Yahoo and elsewhere on the web; then, address book portability. These features soon to me very promising in developing the type of smart applications I am salivating for: automatically sort email based on importance of friends (or perhaps even by work friends versus beer friends, etc.); create "personas" that one can associate with their own activity streams on the web/Yahoo, and associate them (or even perhaps share with) similar friends.

NOW we're talking. I think Yahoo! is particularly motivated to be more aggressive about this and open up into a platform, then others (it's the "we try harder" strategy since Y! hasn't done well lately). For whatever reason, this pushes the boundaries of what's open, and gets closer to my dream (of taking my data, meshing it into others, and develop smart apps that I can carry with me). The only thing that's left is figuring out how to make money on this, in a different way than through the good old eyeball marketing. Truth be told, Yahoo! may also be closer than Facebook at least, particularly given the fact that since Facebook is not that open, the only thing I am left to do with the latter is read/look at pages and pages and pages (read, not read/write).

So Yahoo! this is great, I will start playing with Y!OS (I can hardly wait), but my question is: how are you monetizing this in a way that's more about data and people than pages and eyeballs? Let me know when you figure it out :-)

A Response to Anti-Web 3.0 proponents, and an explanation why Web 3.0 is better than others

Mike Bergman has responded to my rather visceral reaction to his visceral reaction ("squash the "web 3.0" cockroach" - see my original post here: Web 3.0, (semantic) advertising, and why Semantic Web can't get their %&$^ together. This is my response to him, and a clarification for my position pro-"web 3.0".

Mike, thanks for your thoughtful response. I do not intend to start a flame war at all, I reacted a bit viscerally to a visceral reaction as well. We shall agree to disagree of course. I found your response organized, and a viewpoint backed by arguments. Where I come in disagreement is not on the semantic domain (e.g., to your point re "web 3.0" is meaningless), but rather on the pragmatic one (I am hereby referring to the classical syntactic/semantic/pragmatic triad, and specifically Wittgenstein's idea of "language games" - see the theory here. In it, the idea is that the meaning of a word, therefore of a semantic process moniker as well, presupposes the ability to use it). I sense in your position a rather religious/aristotelic/absolutist view of the "meaning" of a concept, etc. I believe this meaning (certainly of both "semantic web" and "web 3.0", "structured web", or "linked data" altogether) is easier understood by enterprise clients, as you rightfully mention. But no one else, certainly not consumers. Therefore, it is not better (pragmatically speaking) than "web 3.0", because of its rather limited ability to use it in language by a limited section of the audience.

The advantage of "web 3.0" is purely pragmatic, and not semantic (not any different than "web 2.0" with which I am sure you also disagree with its usage); the term has been established via public debate and contribution to its denotation, and is now accepted and well understood. Yes, not so much yet about "web 3.0", but the idea, and my goal has been to offer a much better PRAGMATIC term to the community and audience at large, void in the beginning, but with the advantage of; not being loaded with technicalities (not needing the presence of ONLY certain W3C standards but not others, in a very autocratic way); showing continuation of a process from 2.0, and thereby linking it in the mind of the public with a moniker that has been already accepted (a decent starting point). I am neither scared of public debate, not against it - in fact the reality of our debate works to better define whichever term will be ultimately adopted, and made successful to a larger scale adoption of semantic technologies, processes, etc. At this time, I think all terms you mention are doing us a disfavor: too technical, not easy to be understood by non-enterprise clients (and this is what we need the most today, the consumer imagination, because we already have better success on the enterprise side than we have at a larger scale). Yes, they may be more "meaningful", but I don't think that's necessarily the point: I can give a ton of examples of concepts that are very clean semantically, but that never lead to/supported a larger adoption by the general public. I think this is the case with the terms you have been proposing.

Quite frankly, it is my belief that the terms you propose have not been adopted by more than a few smaller communities within the technology community itself, so those are not settled either.

Bottom line: we have been debating on two different "worlds" and philosophies: I don't believe in absolutist/aristotelian/platonic ideas of "meaning" of a term as set in stone by an authority, but I rather believe meanings are co-created in the praxis of language use by people/participants to a democratic exchange of ideas. I also don't think any of the terms currently proposed do us any favor, for two reasons: 1. they are too technical for larger adoption (albeit with tighter denotation); 2. they are overloaded with negative connotations, and have limited scalability.

Lastly, I also do not think that just because "web 3.0" is meaningless, it will never be meaningful (if you consider my pragmatic approach to it): it just means that the public debate has not settled on it yet, as you rightfully agree with. Truth be told, if there is no "higher" authority to shine on us the right meanings of our concepts but ourselves in the practice of language, then any and every concept/term has been meaningless at some point, acquiring meaning in the process of its use.

Yes, I agree terminology adoption is a function of both providers and consumers and has been largely given by providers. I also agree with your comment that "Consumers vote with their attention and their wallets". But I believe consumers have not voted either with their attention, or with their wallets so far, which leads me to believe that they don't care about "structured web", "linked data", etc. Why? For reasons explained above, but also because they shouldn't: technology should in fact be transparent to them, as long as a fundamental need of theirs is fulfilled better with this technology than with another, that is what counts. Not the technology itself.

Web3Beat Dictionary: "monetization"

It daunted on me that a lot of people in the blogosphere associate the term "monetization" to refer to only advertising-related business value (see my previous post). I feel compelled to start a mini-series of a pseudo-dictionary that reminds people that terms have more than one denotations.

Monetization (as in, monetizing the web, monetizing the semantic web, etc. etc.) = "revenue from business operations" (I am sure we can find a lot more scientific or rigorous definitions, but this one from Wikipedia entry on "Monetization" will suffice for now).

In other words, monetizing the web is NOT just about advertising; advertising is one way of monetizing it, another is subscription, etc. Generating financial value through various methods; a type of business value among others (although I can't imagine a BUSINESS can be successful without extracting in any way financial value from its technologies or operations; religions, cults, cult-like businesses, Web 2.0 copy cats without clear business plans, and focus on a user problem, certainly don't qualify as businesses).

So as I heard many bloggers or comments to blog posts recently identifying "monetization" with advertising, let's stop for a second, go back to school (oops, I meant dictionary), and remind ourselves that monetization is NOT just about advertising. Not that there's anything wrong with advertising.

Web 3.0, (semantic) advertising, and why Semantic Web can't get their %&$^ together (because someone has to always pay the bills)

I have just returned from the Web 3.0 Conference & Expo that I chaired two days ago. I am exhausted, but happy it turned out a great success. It has also brought up (as one can see in the recent articles on ReadWriteWeb) some concerns and a very select "technologically stalinist" comments, as I like to call them.

One for instance, reported on the first Keynote, from Amiad Solomon, the CEO of Peer39: Semantic Web" Making Advertising More Relevant to Consumers. The concerns were that the conference, as well as positioning of "web 3.0" alongside advertising, semantic or not, is wrong. A lot of people were surprised. For instance, check the comments in the aforementioned RWW article. I have also shot a few comments, most visceral, as I was both surprised at their surprise, and then shocked at the level of both immaturity and totalitarianism in mentality of certain people associated with the "Semantic Web" community. Let me explain:

First off, I don't understand why semantic advertising is negated by some commenters, in saying that semantic tecnhologies are not/should not be about advertising, but about improving the user experience, content, etc. True, I say. But why (as is implicit on these assertions) advertising should not have a user experience that needs improved? As the argument of these comments goes, advertising is but a sliver from the total market revenues from various sources on the Internet (content, enterprise, etc. being the most). Ok, that explains why Semantic Web has largely been an enterprise IT success (in ETL, data integration, etc.) but not yet in consumer mainstreaming. As I explained previously, Web 3.0 is about customizing/personalizing user experiences, making them relevant, social, in an intelligent digital command center of a user's "digital life" (and perhaps in the future not just digital). That is fine and dandy, but if you agree with that, why advertising needs to be perceived as "evil"? I think lots of people still don't get it or don't want to get it: if you want consumer (not enterprise) adoption, you need (beyond a cool technology that few non-technical geeks can understand their value proposition in non-technical concepts) product marketing, positioning, etc. The stuff that your regular "hippie" IT guy hates: the business and product strategy. I have worked in many a technology company and this is no different than the typical developers/R&D v. the "suits" stupid friction. It seems some people viewed the fact that we started the conference with a keynote on semantic advertising as an attempt to hijack semantic web to the realm of advertising solely. Nothing else more wrong than that, let me just say. So, if we started with a keynote on "semantic puppies", then the conference would have been perceived as an attempt to define web 3.0 as dogs? This doesn't make any sense at all.

Second, the neverending "suits" v. devs friction is stupid and needs to be gotten over. Peer39 is an example of a success in semantic technologies that happens to apply it to advertising, in making it more relevant and its user experience better, compared to other types of advertising. I will be sure to start the next event, in April 09, with a keynote on "semantic puppies" then, to ease some concerns.

Then, some comments on the RWW article mention, beg, bitch, become downright nasty (Please, Squash That Web 3.0 Cockroach) and unprofessional, about the use of versioning the web. These folks' comments remind me of their complete misunderstanding of real business functions (those same ones seem to not realize a few fundamental facts - listed below), and by the contradiction inherent in their arguments: one the one hand they are upholders and data and application freedom, so true democrats indeed, and on the other they want advertising dead (because it's "evil"), they see the entire business/market machinery as wrong. To them, let me just say: aside from the fact that I would prefer reasoned arguments versus diatribes and pontifications that the semantic community has been full of (which contributed to the lack of almost no success in adoption of a killer app, in explaining clearly its value proposition to non-technical folks, such as users, "suits", etc.) - it's more convenient to hide behind the asynchronicity of a blog (versus trying to raise the same bitching within the conference itself, as this will always be an open forum and an attempt to bring success and mainstreaming by collaborative definition); it's also more convenient to deny what you can't understand. And by the way, while largely Semantic Web projects are excellent academic projects, they have not proved to yet be successful businesses. But then again, perhaps building a business is not their intent, and that is ok too. However, someone will always have to pay the bills (for people who enjoy the peace of their R&D office): typically, that has been either the user, or advertisers (through advertising). If anyone has in mind anything else, hats off, that would revolutionize the way markets are moving and have, for the last 200 years +.

In other words, if you hate "web 3.0" so much, you must also hate web 2.0 and web 1.0. And I give you credence, because the web is one, not many, and the user doesn't care about nomenclature, but they also don't care whether the reason an application makes their life better is 1,2,3, semantic or what have you. In the end, it's about improving their experiences anywhere; it's about hiding the seams of technology (making it invisible) behind solving important problems. This is why I don't think "Semantic Web" has potential to generate passion in users: because they don't care about technology (outside of our own circle). On the other hand in order to achieve user success, we need monikers that everybody easily understands: we only have a few seconds attention (because of information overload, a phenomenon Web 2.0 has brought about, which 3.0 wishes to solve). While Web 2.0 has gone through the same rounds of denials, O'Reilly has made it clear that the term is a marketing, not a strict definitional term. And that we need this, if we want user adoption, and business/market success. For the latter, we also need either advertising, or subscription (if we don't just want enterprise adoption, which frankly, is conveniently easier to build since the environment is more controlled and the audience is more technical in nature so they understand tech language). My hunch is that as much as some of us hate advertising (and by expansion, marketing), this is not dissimilar to the neverending research in mass/popular culture sociology on "conspicuous consumption" brought about advertising and marketing. While this may be true, I urge you to try living without advertising/marketing, but if you must hate online advertising and semantic forms of it, then you must also remove/deny any other forms of it, including Yellow Pages, directory assistance, PSAs, etc. I am not so sure how would you be able to find out about products that may interest you then. But then perhaps you don't need products because you don't want to be relegated to a consumer, but want to be a holistic person. That's a great feeling, and some of us marketing technologists are working on new models that humanize marketing and advertising to make it more relevant (Peer39 is one, check my earlier posts for others, etc.). But to throw the baby with the bath water is downright dangerous, immature, and proves basic misunderstanding of what makes markets work.

And that my friends right there, explains how come none of the anti-advertising, "hippy", contenders of Web 3.0, advertising (semantic or not), have been able to explain what they bring to the user and business/market at large as value, and why we still keep having these definitional arguments over and over again. To those, I wish they prove me wrong, come up with revolutionary business models, are successful in obtaining user adoption. The moment they do so, rest assured I will give them the "mic" on my blog and they will have an assured future keynote of the Web 3.0 Conference. Until then, might I suggest: less talkie, more productie? Otherwise put, less complaining, more paying the bills.

Fundamental facts:

1. Users don't care about technology - they care about using apps that improves their lives and experiences;

2. Users don't know what "semantic web" means;

3. Advertising and marketing are not evil per se - poor/irrelevant applications of it are;

4. Business success is based on the right combination of business acumen (strategies, product marketing, sales, business development, etc.) and technology. Not one, not the other, both in tandem;

5. Saying "semantic is not about advertising" is no different than saying "semantic is not about web" - semantic is a technology; it could be applied to the web, but it could also be only applied to enterprise data integration controlled environments;

6. Denying 3.0 to "web 3.0" logically impies denial of "web 2.0" as well;

7. "squash the cockroach" type comments to "web 3.0" misunderstands the role of marketing and product strategy to a technology;

8. We can't advocate data democracy (in the Semantic Web) while denying one application of it in advertising. That's just being inconsistent and contradictory. Frequent visceral commenting on it = technological stalinism/anti-democracy;

I have lost more to say, but am getting tired and repeating my beliefts over and over. It is clear to me that some members of our community have an overall different perspective on life, technologies, business, etc. and a visceral one at best (myself included). The one thing I have been trying to do is help our community not "come together in harmony to a same definition" but provide a forum for "agreeing to disagree". But when I hear comments such as the visceral ones above, I can only think of the burning of books in Nazi Germany. Not nice, not nice at all.

R.I.P. Web 2.0 (especially the copy cats)

Copy cats watch out (and I mean those 100/day new startups whose big idea is: "Twitter for cats", "Facebook for dogs", etc. etc.): Mike Arrington's post confirmed my thoughts (Audio: play "Don't Stop Believing" track from Journey a la Sopranos finale) - Web 2.0's year of final gasp is 2008! In other words, bad news (for those of you with minimal long term business planning, and the hacks): the current economic situation demands us to go back to work and stop building countless of useless apps with no monetization potential ("Twitter for dogs" anyone?). Good news: my job of tracking the landscape (I've been keeping countless information in a SWOT analysis and market landscape of all these for my own reasons but was always pissed with so many useless pretty sites popping up every day, with the sole intent to exit within 1 year or so taht I could barely keep up with it) has just been made easier.

Did you think that bypassing college and doing hacks with no understanding of system architecture and/or a Comp Sci degree would take you to the stars? (well it did for a while, but that's because our markets got giddy like school girls, but what goes up must come down). So for those of you who were debating whether to stay in school or quit and start a Twitter for dogs: DON'T!

The way I read the current economic situation (not that I am not concerned): finally, a fresh of cold air to wake us up and focus on what's important, not on what's pretty and useless. Can you say Web 3.0? Let's make sure we really use our down time to solve real consumer and business problems, not hack a bunch of code, pretty it up and hope someone will acquire us in 12 months. Good stuff indeed!

How to Solve the Fundamental Open Data problem

I had a brief email conversation with Jamie Taylor, Minister of Information at Metaweb, and he framed the issue of open data from within a behavioral economics perspective: expected utility theory. I never thought of this but had been on the look for a food (or at least better than the pundit blogs that are pro or con on S(s)emantic application methodology or another). I think I have found it, via Jamie, whom I thank for opening my eyes. I will write more on this tonight and attempt to establish the value function and its parameters, because I think being rigorous at this will set us straight on being able to "sell" the 3.0 story in the marketplace. Here I go:

There is a fundamental problem/conflict between user needs (e.g., I want all my data to be mine, portable and across all data providers/sources/sites, etc.) and data owners (big Internet companies, sites, networks, Walmarts, Amazon's, your grocery store, your medical insurance, anyone holding, processing, doing something to the data you leave behind in your behavioral trail of interacting with them). The data owners must be convinced (in a granular fashion, by answering every parameter in the "expected utility function" equation, e.g. size of payout (to data onwers, of opening up their data to developers/users), probability of occurence (of this payout, which is essentially our business models of Web 3.0 app providers), risk aversion (check the panel Business Risks of Web3.0: What Risks? at Web 3.0 Conference & Expo next week), and account for differential utility of the same payout to different companies/verticals with different assets and/or needs.

That is the question that needs answered. If we continue to talk tech mambo-jambo that the market doesn't understand, but don't have pretty damn good answers for the above, we're screwed! The Web 3.0 Conference is an attempt to get us all straight.

A Web 3.0 Challenge: help me concept-ualize this blog!

In my blog, I really like Zemanta's semantic features but what I'd really like is: instead of linking to relevant documents on a specific term/concept, why can't I embed links to relevant concepts? Whether using dereferenceable URI's via W3C standards, Linked Data methodologies, and/or any other methodologies. This is a challenge for any individual, company, etc. to help me achieve this personal dream. Can you help? In doing so, keep in mind that I won't be able to code the site myself, so minimal metadata tagging is possible (although not completely impossible), so given this, who can propose a solution to my "itch"?

By the way, isn't this one nice use case for Web 3.0?

Top Down v. Bottom-Up in the Semantic Web, aka Logic v. Science ..... again!

Greg Boutin has posted an excellent response to Richard MacManus's post on Where Are All The RDF-Based Semantic Web Apps?, which you can find at RDF/Linked Data Standards Not Good Enough For Intelligent Agents? Or Is It The Opposite?.

In it, Greg asks the question:

"does RDF serve intelligent agents properly? Because the point here is that, if it doesn't, then we need an alternative method to get the job of representing information done. Intelligent Agents alias Top-down technologies simply can't do without it. HTML, XML, Relational Databases don't seem to cut it. So what does? Do we need a completely fresh approach, or changes to the existing RDF stack / Linked Data model? Those are questions I ask genuinely. I don't know the answer."

Let me try to answer based on my personal experience (still ongoing) while thinking at these issues from a practical perspective of working on an intelligent application that uses semantic data (notice the lower case). And by the way, let me explain where I'm coming from so that you understand the context of this post, and the fact that I am not in the business of technology politics (upper versus lower case semantic, and/or top-down versus bottom-up), but rather I've studied it extensively from the perspective of someone who has problems that need be solved without waiting for an entire technology to come to mainstream fully:

1. I am a "science" guy, that is: my background is in Data Mining/Machine Learning, not RDBMS, etc. (although I do get the depth of metadata having had to work with different over time in different situations), and to me, "intelligence" = algorithmic; I need my "intelligence" to do something to the data that was not there obvious before (read: Information Extraction, Context Extraction, recommendations, personalization, targeting, etc.);

2. I do like the idea of the semantic web and have been fascinated by it since I discovered it: I think it's both a much closer way to structure data to the way we humans think, it's more flexible (albeit not scalable enough with current technologies) than flat, or relational ways, and I do believe the marriage of ML and semantic structures is made in heaven.... theoretically at least (until someone comes up with a good app that needs to fulfill the following conditions: is consumer-facing, is solving a real problem and not built as showcase for a specific technology, has a business potential to it (read: business model), and is evangelized in a consumer-friendly way (not by a bunch of technical blurbs that I may understand but my wife certainly wouldn't).

Given these pieces of information about me, I wanted to say as response to Greg's post that :

1. the key to your question, Greg, is having a shared and clear definition of "intelligent". I see "intelligence" as described above, while most apps I see our there (RDF-based or not) use "intelligence" to showcase a slightly better version (usually in SPARQL) of the typical "SELECT .... WHERE" from SQL. To me, that's not even intelligence. It's certainly useful, but it speaks more about data interoperability, not intelligence. I need more than that. By the way, the distinction I have made is not different from the one encountered when people use the word "analytics" or "Business Intelligence": most refer to "reporting", but very few mean the concept denoted by activites such as "predictive modeling, (un)supervised clustering, Bayesian nets" etc etc. So let's just say I am using a stricter definition of the term "intelligence".

2. From this perspective, I think that the type of "intelligence" that I mentioned before has long been used in Machine Learning (see Bayesian Networks, and others) in a very graph-based way (albeit not with RDF), that is: processing and generating intelligence by leveraging relationships/links between terms/concepts/attributes, etc. This is really no news for me. The only difference is that the aforementioned methodologies have not used (until recently) data physically structured in a graph-based form, but rather using XML-based "rules" (or external application logic) based on relational or flat structures. So to answer your point, this stricter definition of "intelligence" was quite ok without RDF.

3. More recently (and this is an area I've been working in, and doing research), there are VERY very few attempts to use (my stricter definition of) intelligence directly on physically structured graph-data, and I could name some:  SPARQL-ML, Proximity, the work in CRF specifically by Andrew MacCallum (Mallet) and other open source, academic projects. I know of some efforts of doing similar things by various startups in a very proprietary, but practical way too. The technologies cited above are all part of a larger area (that mixes modern Machine learning technologies with the good old Prolog-style Logic Programming) of Statistical Relational Learning ("relational" here does not refer to RDBMS, but stands for "relationships" between concepts/attributes, etc).

To your question, I would say that "intelligent" applications are ok with whatever way data is structured in, but that if they use graph-based data/metadata (even with OODBs using Hibernate on top of MySql), they have particular advantages by doing so above and beyond doing it on top of flat or relational data. But they are quite intelligent enough for semantic data, I don't see the problem as one of technological misalignment, but rather both practical and political. Let me explain (using my experience as I promised before, and without necessarily giving concrete details quite yet :-):

A. I have tried to use RDF for my "intelligent" applications, but it would have taken way too long, would have had major scalability complexities, and I approached my problem (in trying to build an intelligent app on top of graph-based data) looking for a solution rather than viceversa; in the end, and at this time, I decided to go a specific custom route, but not taking advantage of the full RDF (I guess that would make me a "top-down" although I agree that Alex Iskold's distinction seems the opposite of what I would think of being rather "bottom-up"). Everytime I tried to implement RDF at the bottom of my "intelligence" I would get either "you need to change your intelligence" (which would have substantially reduced my intelligence to "SELECT .... WHERE"), or "it's not quite scalable yet" (given that I need to process floating point operations at run-time on millions of rows but traversing the graph in a very depth-first search way) that I didn't think is possible quite yet. I would get these statements from avid proponents of RDF that did not quite understand my ML meaning of "intelligence" and would try to reduce it to "but do you really need that? Why can't you take Dan G. is_friends_of from one source and link it to Dan G. likes_beer from another?"). To me, the example here is what all RDF-based apps are able to do today (and for some time I guess), but not my specific application of "intelligence".

B. Political, because everytime I would genuinely try to solve my problem is really do due diligence on most optimal technologies that would allow me to do that, I would get "top-down" suggestions that I really should use RDF, etc. etc. Which to me, is technological totalitarianism. It looks from the solution's standpoint, in search of a problem, not otherwise. I mentioned before in earlier blog posts and articles that this is the major problem Semantic Web faces today.

In the end, I have a problem that needs solved (I solved it, and think I know how), and the key answer to your question (and another source of the problem) is that there are two "schools of thoughts" and behavior when people talk about "intelligence", etc.: one that is heavily warehousing-based (whether the warehouse is relational, or of any kind of flavor) (and I call this the logic-based guys), and the other is the algorithmic guys (I call these the "science" guys). There is very little common understanding of each other, sadly (because the two are really interdependent as you mention), there is even less shared "hanging out" in joint projects that leverage each other. Let me give you an example of what I mean: one time I've asked the "logic guys" (these are the typical RDF, but ex-RDBMS guys) to give me a randomized sample of some Internet data (that was extracted from various sites); I got it, and thinking it's truly random, I started my algo-type work, only to figure out it was nothing random in it. When i talked tothe logic guys, they told me "well, sure it's random, the database was partitioned at random daily". To which I asked "wait! how was the partitioning done?". The answer was: "well every day, we'd take the first 1/2 of the daily data and put it in one partition, and the rest in another/others". Clearly, we were using the same concept, but meant opposite things. This is a clear example of how logic and science guys don't talk "nice" to each other.

I think there is an asnwer to this, but I think it has more to do with practical and political reasons than to technological problems. What say you?

Ditching the Self-delusion of the Semantic Web

Just came across this post Ditching the Semantic Web, and I must say: BRILLIANT! It hits headfirst at the problems that we have. Well, I'd like you to first read this, then come back tomorrow for my next post, arguing in detail on why and how we should ditch the self-delusional (and already institutionalized) Semantic Web.

Are you with me?

On Data Ownership, Cloud Computing, and other Scare Tactics

Just read Richard Stallman's reaction to cloud computing here: Cloud Computing Is A Trap, Says Richard Stallman. Following shortly after Larry Ellison's rant against cloud computing, this made me think of the cloud computing opponents from the perspective of data ownership and/or privacy,which is what Stallman's position originates from.

Well, basically I think their position is untenable, because:

1. if we follow rms's proposal to stick to our local computers for all data processing, application usage, etc. then the whole Internet is useless; because we are leaving a "data trail" wherever we may roam online whether we like it or not. Not different from leaving a video trail as we browse through the brick and mortar aisles of say, Walmart. So this position is absurd, because by the same token, we should not use phones (in fact Stallman shunning cell phone usage personally for that same very reason), or any other technology since the Industrial Revolution (since really, the start of the use of data and statistics for demogrpahic purposes initially, but then expanded in all areas of life, started with the Industrial Revolution). So in other words, let's go back to the grotto.

2. The alternative is basically informational solipsism: stick to our own local version of the universe (read: computer), and own our own Internet. But wait! I am confused: then how does this go hand in hand with open source projects that are benefiting from Internet-enabled collaboration (e.g., Linux, Wikipedia, etc.)? This makes no sense. As many of rms's rants, they're more emotional than reasoned. Or maybe I am missing something.

3. There is nothing wrong with cloud computing, IMHO. No technology is "evil" per se, but only how it's used can potentially be (what are we, a bunch of luddites?). The fact that we leave a "data trail" behind us does not mean handing our data (is it really ours? more to come on this specific topic) to third parties, but it's an inexorable aspect/effect of the digital life. Can we do without? I don't think so. Here's a better alternative: own or co-own your own data (BYOD=bring your own data) as the Data Portability Project is doing (albeit with limited mass success mostly due to the lack of, and difficulty of a MASSive user/consumer advocacy campaign).

So, I will return to the aforementioned topics in the next posts because it's something i think passionately about myself, but here's something to have you "talk amongst yourselves" until I come back with more of my thoughts; I'll give you a topic:

A. How do we "take back" our data from those "evil" third parties? Answer please. And while you're at it, don't forget that it just so happens that the reason why it's so difficult, is that until the advent of the concept of Web 2.0 and 3.0, the data layer was ONE with the presentation layer, in other words, "our data" was glued to the channel/conduit in which it happened to appear/travel, etc. So, up until now, very difficult to "take it back". Now, not so much. More about this in my next posts, and a lot about these issues at the Web 3.0 Conference & Expo, October 16-17, Santa Clara, CA. Come see me there!