Blog Post

Search engines, big data and network effects

Search engines are intermediaries in a two-way market between users and advertisers. Their huge stocks of data about users and their preferences can help search engines offer better services to all parties. But does this make market entry difficult for new players? And can we see network effects emerging in the search engine market?

By: Date: November 22, 2016 Innovation & Competition Policy Tags & Topics

Search engines respond to queries by providing relevant and valuable information about the topics that their users are looking for. They also act as intermediaries which match consumers with providers of services or sellers of products. To monetise their work, search engines collect and process data from users and sell advertising slots to companies. By analysing the users’ data, they can improve the quality of the search engine algorithm and provide more relevant organic search results, but they can also design personalised advertising strategies for companies’ products and services. Companies are thus more successful in placing their products, and consumers receive recommendations tailored to their interests.

Advertising slots are allocated on a competitive basis to the companies that are willing to pay the highest amount to get some prominence in the platform when the user is searching for relevant terms. Slots are auctioned and the winners have to pay a fee to the search engine whenever a potential consumer clicks on the displayed ad (pay-per-click strategy). Search engines base their revenues almost exclusively on selling advertising slots (see for example some relevant statistics in Liem and Petropoulos, 2016).

So, in this two-sided market environment, advertisers assign a higher valuation to participating in the search engine platform when they expect to have a higher probability of matching with as many consumers as possible. More consumers can increase the value for companies to be advertised in the search engine in two main ways. First, companies obviously see that they can be connected with more potential buyers as the pool of relevant consumers increases. Second, search engines with more users can analyse the additional data and improve their matching algorithms. They can then provide more personalised services to advertisers which can in turn have higher chances of success in attracting consumers. Given the competitive allocation of advertising slots, more consumers, therefore, implies higher revenues for the search engine as advertisers are willing to bid more for prominence.

Consumers, on the other side of the market, value information that leads them directly and quickly to what they are looking for, to minimise their search costs (for example, consumption of time). Hence, they mostly prefer to only visit the search engine of the highest quality (according to their standards and criteria) even if they have the option to use different search engines simultaneously.

In betwen these two sides, search engines compete in quality and efficiency of search (for example, accuracy of results, page load speed, real time relevance) in order to attract more consumers who will in turn bring more advertising revenues which can be used for further improving their quality to attract even more consumers, and so on. So, it is of interest for the search engines to build up a large stock of users’ information that can be analysed in order to provide good quality services which attract more consumers and advertisers.

How important is the scale of data collection for the improvement of services?

In general, the answer depends on the complexity of the algorithm in place, the number and the noise in observations. But in the context of data related to human behaviour and preferences, the amount of data and the quality of predictive power seem to be positively related.

When data are drawn from human actions, noise rates are often high. Social scientists have long argued that one way to circumvent the poor predictive validity of attitudes and traits is to aggregate data across occasions, situations, and forms of actions. This provides an early suggestion that more (and more varied) data might indeed be useful when modelling human behaviour data. The implication for predictive analytics based on data drawn from human behaviours is that by gathering more data from individuals (aggregated by the modelling), one could indeed hope for better predictions.

Fortuny, Martens and Provost (2013) demonstrate that when predictive models are built from sparse, fine-grained data—such as data on low-level human behaviour—economies of scale are important and we continue to see marginal increases in predictive performance even to very large scale. However, the curve between predictive power and data size do seem to show some diminishing returns to scale, albeit without any ceiling that limits the increasing pattern.

Their empirical results are based on data drawn from nine different predictive modelling applications, from book reviews to banking transactions. The study provides a clear illustration that larger data indeed can be more valuable assets for predictive analytics similar to those used in search engines.

Big data and quality driven competition

Since the quality of search engines increases with data gathering and analysis, search platforms with a larger stock of information than their competitors can have a competitive advantage. They can perform better and attract more users and consequently more advertisers. This suggests that the amount of data processed is linked with market power in the search engine market. Incumbents with a large stock of information and experience in data analytics can protect their market position against new entrants and firms that are far behind in data accumulation. Moving to the frontier of the search market and establishing a large base of users requires new entrants to offer very attractive prices for the advertising slots and at the same time to bear the costs of developing a good quality dynamic algorithm that can attract consumers. This is an investment that can only pay off in the long run. Bing, for example, in its effort to increase its installed base of users incurred losses for many years before starting to be profitable.

Note that in many markets product differentiation among competitors is multi-dimensional. Consumers’ preferences are affected differently by each of the multiple dimensions. If I need to subscribe to a mobile operator my choice depends not only on the quality of the network but also on whether I want to combine mobile with TV and home internet, whether I wish to call often in another country or how much mobile internet I intent to use. The plurality of options and preferences lead to multi-dimensional market competition where there is room for entry and growth of firms that specialise in specific dimensions.

The search engine market is by construction centralised and product differentiation has only on dimension, the quality of service. (Some could argue that privacy is another dimension, but this is a secondary choice parameter, especially since it is difficult for the users to rank competitors with respect to the privacy protection they provide). That means that the search engine of higher quality than its rivals can increase its market power relatively fast or even dominate the market, especially under the implementation of efficient methods of machine learning and data analytics.

Moreover, in many cases, companies that own a popular search engine can use the collected data for their operation in other data-driven markets to improve the quality of their services and consequently their market position.  For example, since 30% of mobile searches are about location, queries in the search engine about finding a particular location can improve the quality of navigation services through the additional data collected by the search engine. So, the search engine can help the “mother” company to enter to new markets and improve its market position through data accumulation and analysis.

Are there network effects in the search engine market?

Direct network effects in an online platform exist when an increase in the number of users or in usage of the platform directly increases the users’ value from accessing the platform. Facebook is a typical example of a platform where the value of using the network highly depends on the number of friends that also use the network. In search engine platforms, on the other hand, such direct effects are absent as each user derives individual benefits from its use without any direct effects to other individual users.

Indirect network effects exist when the increase in the number of users or usage of the network spawn increases in the value of a complementary product or network, which can in turn increase the value of the original. The relevant question is if there are indirect network effects in the search engine market which increase the value of participation for users.

Hal Varian, the chief economist at Google, argued at  a recent Bruegel event on big data and market competition that it is the accumulated experience in the market that helps search engines to learn how to improve the quality of their services (learning by doing effect) just as it happens in normal businesses. He claims there are no indirect network effects involved on the users’ side:  “The higher the number of customers a business has, the higher the revenue of the business, revenue which can be reinvested in the maintenance and improvement of the business so as to attract more users.”

While experience clearly matters, the impact of learning by doing effect is greater than in normal businesses due to the importance of data in improving the search engine algorithm. The addition of new users makes the search engine more competent to provide high quality services through the processing of the additional data collected by the newcomers. That generates extra value for the advertisers and further increases search engine’s revenue as well as its incentives to improve quality and attract even more consumers.

However, I would argue that indirect network effects are also present. We should not ignore that search engines are two-sided markets in which both sides are inter-dependent. What happens on one side has an impact on the other side and affects the value of accessing using the search engine for both sides of the market. The addition of new users increases the value of using the search engine for the existing ones not only because of the increase in quality (due to the learning by doing effect) but also because more users attract more advertisers. That leads to more matching opportunities for the existing users through the sponsored links of the search engine and therefore their value from accessing increases.

Network effects may also extend to organic search results given that the ranking of firms depends to some extent to firms’ choices such as their search engine optimisation strategies. While the algorithm of the search engine is secret, published guidelines provide an overview of how firms can increase their chances for high rankings in the organic search results. The arrival of new users increases the incentives for firms to get a better ranking position in order to attract more users by investing in search engine optimisation and the quality of their website. So, they provide better web services to their visitors (existing and new) and help them to reach what they are looking for in a more timely and efficient fashion.

When we assess market competition in the search engine market we need to take into account the importance of big data in defining market power. Especially when users do not pay any price to access search platforms and the latter compete in other key dimensions such as the (data-driven) quality of service.


Republishing and referencing

Bruegel considers itself a public good and takes no institutional standpoint. Anyone is free to republish and/or quote this post without prior consent. Please provide a full reference, clearly stating Bruegel and the relevant author as the source, and include a prominent hyperlink to the original post.

View comments
Read article More on this topic More by this author

External Publication

9783319418445

Policy and Politics in the Era of the Industrial Internet: How the Digital Transformation Will Change the Political Arena

The digital transformation has already had an impact on policymaking, and this trend will continue in the years to come. How will the political process change and how can influencers guide this change?

By: Giuseppe Porcaro Topic: Innovation & Competition Policy Date: December 7, 2016
Read article Download PDF More on this topic

Policy Contribution

pc-22-16

Going local: empowering cities to lead EU decarbonisation

Decarbonisation and digitalisation are reshaping the European energy system, which will become more decentralised and interconnected with other sectors. Cities have the opportunity to be the key drivers of decarbonisation, but this will require the implementation of a new bottom-up governance system. This paper outlines a four-step mechanism in order to achieve decarbonisation at city level.

By: Simone Tagliapietra and Georg Zachmann Topic: Energy & Climate Date: November 30, 2016
Read about event More on this topic

Past Event

Past Event

Digital platforms: A policy and research agenda

The number of digital platforms is currently rising in many countries and sectors. What are the opportunities of platforms and which kind of regulation and policy framework is necessary to promote healthy competition?

Speakers: Bruno Basalisco, Diane Coyle, Jacques Crémer, Werner Stengg, Nicolas Petit, Georgios Petropoulos, Bruegel and Simon Wilkie Topic: Innovation & Competition Policy Location: Bruegel, Rue de la Charité 33, 1210 Brussels Date: October 20, 2016
Read article More by this author

Blog Post

Giuseppe Porcaro

The industrial internet will transform policymaking

The ‘internet of things’ will bring major changes in many areas of life, including the political arena. What will be the new communication tools, strategies and narratives for policymakers?

By: Giuseppe Porcaro Topic: Innovation & Competition Policy Date: September 28, 2016
Read article Download PDF More on this topic More by this author

External Publication

jsm-ext-paper-cover

New network neutrality rules in Europe: comparisons to those in the U.S.

This paper explains the similarities and differences between European and U.S. net neutrality rules.

By: J. Scott Marcus Topic: Innovation & Competition Policy Date: September 15, 2016
Read article More on this topic More by this author

Blog Post

Silvia Merler

The Apple of Discord

What’s at stake: On August 30th, following the results of an in-depth state aid investigation started in 2014, the European Commission concluded that Ireland granted undue tax benefits of up to €13 billion to Apple. The decision is based on state aid grounds: the Commission argues that two tax rulings issued by Ireland effectively granted Apple preferential treatment, which amounted to state aid. The Commission ordered Ireland to recover up to €13 billion (plus interest) from Apple, but the decision is controversial and opinion differ as to the effects it will have. We summarize reactions.

By: Silvia Merler Topic: Innovation & Competition Policy Date: September 12, 2016
Read article Download PDF More on this topic

Working Paper

WP 03 2016

Challenging prospects for roam like at home

In 2015 the European Union adopted new rules seeking to implement a roam like at home regime for member states. This Working Paper highlights challenges in implementing roam like at home, and it provides insights on the economics of international mobile roaming.

By: Georgios Petropoulos and J. Scott Marcus Topic: Innovation & Competition Policy Date: June 15, 2016
Read article More on this topic More by this author

Blog Post

Scott Marcus

New EU net neutrality guidelines are a pragmatic next step

The new guidelines issued on the implementation of European net neutrality rules by national regulators are sensible and pragmatic.

By: J. Scott Marcus Topic: Innovation & Competition Policy Date: June 8, 2016
Read about event More on this topic

Past Event

Past Event

Internet taxation: challenges and policy recommendations

As the economy moves online, it becomes more difficult for national tax authorities to collect revenue. How great is the impact, and what should corporate taxation look like in the digital age?

Speakers: Francis Bloch, Caroline Edery, Dmitri Jegorov, Helena Kiurusalmi, J. Scott Marcus and Georgios Petropoulos Topic: Innovation & Competition Policy Location: Bruegel, Rue de la Charité 33, 1210 Brussels Date: June 7, 2016
Read article More on this topic

Blog Post

Scott Marcus
IMG_20151119_103626

European e-commerce needs better visibility into cross-border delivery prices

Consumers, retail shippers, and European and national regulatory authorities could benefit from enhanced visibility into the price of shipping goods across borders in Europe.

By: J. Scott Marcus and Georgios Petropoulos Topic: Innovation & Competition Policy Date: May 25, 2016
Read article Download PDF More on this topic

Policy Contribution

E-commerce in Europe: parcel delivery prices in a digital single market

E-commerce in Europe: parcel delivery prices in a digital single market

The expansion of e-commerce, a substantial growth opportunity for Europe, is hampered by high cross-border parcel delivery prices. This paper analyses the economics of cross-border parcel delivery and it draws a comparison with the telecommunications sector.

By: J. Scott Marcus and Georgios Petropoulos Topic: Innovation & Competition Policy Date: May 25, 2016
Read about event More on this topic

Past Event

Past Event

E-commerce in Europe: Lessons for parcel delivery from electronic communications

Bruegel was pleased to welcome Andrus Ansip, Vice-President of the European Commission and European Commissioner for the Digital Single Market. He discussed the place of parcel delivery in the DSM, and commented on research by Bruegel scholars suggesting there are useful parallels with telecommunications interconnection and roaming.

Speakers: Andrus Ansip, Jean-Paul Forceville, Annegret Groebel, J. Scott Marcus and Guntram B. Wolff Topic: Innovation & Competition Policy Location: Bruegel, Rue de la Charité 33, 1210 Brussels Date: May 3, 2016
Load more posts