Danny Ayers thinks people have problems with the SemWeb because it violates their expectations.
My guess is that expectations are relative to the problems we want to solve. Particularly, we have an intuition about the order of magnitude of the complexity of our problem and the likely complexity of the solution.
Could it be that what's uncomfortable about the SemWeb is that it's essentially trying to push you to solve a bigger problem than the one you might be thinking about. Ie. not just "make data from my program available for whoever wants it", but "produce a definitive data-model of this kind of information"?
This would be the biggest challenge to expectations, that it redefines the scope of the problem.