Wednesday, June 11, 2014

How much competition is enough?

The topic of competition is a very interesting one and with layers and layers of complexity. While I firmly believe that the path to an Open Internet goes through building competition at the broadband level, and not through complex network neutrality regulations, the issue is not simple.

On the one hand, we have the example of UK, where OfCom has kept a pretty much hands off approach to Network Neutrality, the result has been a very open Internet that is market and competition driven. Almost every consumer in the UK has access to 4 ISPs that are similar in terms of capabilities and the Internet has remained "neutral". On the other hand we have the example in the US, where although significant portions of the nation have at least two comparable broadband providers (say, metropolitan DC and New Jersey where Comcast and Verizon FiOS are widely available) the "openness of the Internet" has been a problem. Specifically there have been peering disputes between Netflix and broadband providers, or the CDNs employed by Netflix and broadband providers. Netflix and Level3 have been complaining loudly that this is coercion from the ISP and quality is being deliberately deteriorated to extract a toll from the content providers. So the question is - if 4 competitors provide Internet openness in the UK, why do 2 providers fail to do so in the US? What gives the broadband providers in the US the confidence to flex their muscles, whereas similar profit oriented entities in the UK don't?

The answer to that involves a lot of factors. First off is the question of long term contracts - in the UK they are prohibited as far as I know, whereas in the US they are commonplace. This makes "competition" less than perfect. Secondly, since the broadband providers in the US are also invariably cable providers, there are vertically integrated monopolies in the US that are absent in the UK. There are many factors that make an ISP sticky for a consumer.  However, the point that I wanted to bring across in this post is what happens when there is "true" competition  - is there a magic number of competitors (4 vs 2) that ensures openness and "good behavior" by ISPs? Isn't 2 providers competing the same as 4 providers competing?

The answer to that is no. Competition is monotonically better for consumers. 3 providers competing is better than 2, and 4 is better than 3. And this is with the assumption that there is no collusion etc. happening with a smaller number of competitors, i.e. the best case scenario for competition. In a prior work, we applied cooperative game theory techniques to analyze the Internet ecosystem:

Richard T.B. Ma, Dahming Chiu, John C.S. Lui, Vishal Misra and Dan Rubenstein, On Cooperative Settlement Between Content, Transit and Eyeball Internet Service Providers, Proceedings of 2008 ACM Conference on Emerging network experiment and technology (CoNEXT 2008), Madrid, Spain, December, 2008


The associated talk we gave is here, but let me try to explain some numbers around competition that can explain the behavior we have observed.


An important concept of cooperative game theory is the Shapley value, which is the share that an individual gets of the value generated by the coalition it is part of. Shapley value incorporates various factors like the value an individual brings to the coalition, the value of the coalition without that individual etc. and the end formula gives guidance on what a rational individual would do to maximize it's share under all scenarios. Our work has the following formula of the “fair share” (the Shapley value) of the class of providers in the economic ecosystem of the Internet:

Let’s say there are m content providers and n ISPs. Then the share of the value generated (V) for the content providers is n/(m*(n+m)) and that of the ISPs is m/(n*(n+m)).

Let’s say Netflix is the sole content provider (m=1) at one end and there are 2 ISPs competing for customers (n = 2). Then Netflix’s share is 2/3 and each ISP is 1/6. So for every say 12 dollars a customer generates for Netflix every month, the ISPs have reason to believe that they deserve 2 dollars of it, because Netflix's business wouldn't exist without the ISPs (I am deliberately leaving out arguments of broadband being a utility that every consumer has the right to etc.).

If now the number of ISPs competing becomes 3, the Netflix share becomes 3/4 and the ISP share is 1/12 each. If you move to 4 ISPs, then Netflix share becomes 4/5, and each ISP is 1/20. Now for the 12 dollars that Netflix generates, the ISPs believe they deserve 60 cents from it.

So moving from 2 to 4 the “rightful share” of the value generated reduces to 1/20th from 1/6th - it is plausible that it is not worth it at that point to play hardball and extract that revenue from Netflix and instead the ISPs are more interested in winning and keeping customers. Maybe at 1/6th (2 ISPs competing) it is worth it to lose a few customers if you end up extracting more from the content provider but increasing the level of competition reduces the utility of that tactic. At some point the expected payoff (of toll) from the content provider falls below the expected loss (of customer revenue) to competing ISPs and at that point "competition is enough".

So more competition is better for consumers, and the cooperative game theory analysis provides some numbers to reason about how much better.