This is the first story in our investigation trilogy titled Facebook Algorithmic Factory, created with the intention to map and visualise a complex and invisible exploitation process hidden behind a black box of the World’s largest social network.

The three stories are exploring four main segments of the process:

Data collection – Immaterial Labour and Data harvesting
Storage and Algorithmic processing – Human Data Banks and Algorithmic Labour
Targeting – Quantified lives on discount

The following map is one of the final results of our investigation, but it can also be used as a guide through our stories, and practically help the reader to remain in the right direction and not to get lost in the complex maze of the Facebook  Algorithmic Factory.

With 1.6 billion active users in 2015, Facebook is heading towards fulfilling their mission to connect every person on this planet through their social network. Zuckerberg’s vision, which is becoming reality, most people on the planet are connected between each other through one application, the social networking app. According to its creator, Facebook was built to accomplish a social mission – to make the world more open and connected. To be fair, this social network has in fact made the  lives of billions of people  more open and transparent, and made segments thereof more exposed to the public.We are the witnesses of the time of transparency of the individual. At the same time, Facebook, the platform itself is far from being open and transparent. What happens within the invisible walls of this complex algorithmic machine mediating the communication of billions of people is kind of mystery, a black box .

There are many reasons why we should be interested in these black boxes mediating and recording our interaction, our deepest personal communications, our behaviour and activities. Within those invisible walls, in every moment algorithms are deciding which information will appear in our infosphere, how many and which of your friends will see your posts, what kind of content will become part of your reality and what will be censored or deleted. 

On other hand, this black box has defined new forms of labour, exploitation and generation of enormous amount of wealth and power (17.93 billion dollars in 2015) for the owners of this invisible immaterial factory creating a deep economic gap between the ones who own and control the means of production and the  users who often live below the poverty line.

Somewhere deep under the layers of algorithmic machines there can be hidden new forms of potential human rights violation, new forms of exploitation and mechanisms of manipulation on a large scale influencing billions of people each day.

Those are the raw resources exploited within Facebook Factory.

Where Surveillance economy meets immaterial labour

According to the Marxist theory, when creating a good, people operate on the subjects of labour, using the instruments of labour, to create a product ¹. The means of production include two broad categories of objects: instruments of labour (tools, factories, infrastructure, etc.) and subjects of labour (natural resourcesand raw materials). For example, in an agrarian society the means of production are the soil and the shovel. In an industrial society they are the mines and the factories, and in the knowledge economy the offices and computers.

If we try to understand the  production process and creation of products at the Facebook factory in this context, we come to conclusion that there is one important difference. The main raw materials in the process (data, content and metadata) are the objects of labour and they are created by humans, but the labour itself is performed by algorithms.

So basically whatever we do on Facebook can be described as some form of free digital labour. According to Trebor Scholz and Laura Y. Liu, ”the instruments of digital labour are indeed everywhere; they are fast-changing and invisible. Without being recognised as labour, our location, input, and tracked mobility become assets that can be turned into economic value.”

Every one of over 1 billion Facebook users, digital workers, work averagely 20+ minutes per day on liking, commenting, and scrolling through status updates.  That is more than 300.000.000 working hours of free digital labour per day.

We should be clear that the main products of the Facebook factory are not billions of texts,updates, uploaded photos or videos. As we will explore in our investigation they are just a resource, playground for algorithmic social network analysis, classification and algorithmic profiling.  Looking from anthropocentric perspective we like to put our self and our labour into the main focus, but in this case, the main form of labour is done by the algorithms. Products of this immaterial factory are more than a billion different user profiles, categorised and ready for sale. Specifics of this system is that users that are being used as a raw material are constantly working on fine tuning of themselves as a target, feeding this system with more and more information about themselves. It is kind of perfect marriage between free immaterial labour and surveillance economy.

Inside Facebook algorithmic factory

In our first part of investigation we will try to put a light upon how our behaviour, actions and information collected, stored, analysed and finally transformed into the products.

Our approach in mapping this invisible system is to find all the inputs and outputs and then try to describe what kind of actions were performed in between.

Our methodology consists of using different investigation tools on the publicly accessible resources. Three main parts in this research concern the Facebook data collection, its storage and analysis, and the targetingtypes applied to the users:

Data Collection : Extracting data from the biomass

Data harvesting, extraction of data from the biomass in the human fields is an essential operation, one of the foundations of the Facebook Empire.

Human fields from the Movie “matrix” (1999)

According to our investigation Facebook utilises different ways to extract data from our behaviour and activities within and outside of the Facebook domain that we can separate in following groups :

A.Within Facebook

Every like you make, every step  you take, every photo you upload, every event you attend, is recorded and stored by Facebook, in their databases. We can separate two main categories of information collected within the Facebook domain. First there are all the interactions, created or uploaded content, pages visited and basically everything you do on Facebook – Activities and behaviour.

We could perceive the second group as rather voluntarily provided content – all the information you provided about yourself in the Profile information segment.

There is a significant difference between those two groups of information. Profile information are basically static information that are rarely updated and depend on the direct input you (are free to) give, on the other hand activities and behaviour inputs are dynamic and represents what you like, share, create and interact with in real time. Profile information can quite often contain misleading or faulty information.

B.Digital Footprint

Digital Footprint harvested from your devices is the second great resource of information Facebook has about. In this case we have two main categories as well: Information that can be gathered from your mobile devices  and digital footprint that can be collected from laptop or desktop computers you use to access Facebook.

Information that can be gathered from laptop or desktop computers are not as diverse in comparison to information that can be gathered from mobile devices. However, they can still reveal a lot of information about you. Some of those information include your IP address, operating system, browser type and other information that can be used as a unique identifier and combined with information gathered through cookies and pixel technology reveal different behavioural patterns.

Cookies

Facebook is not gathering information just within Facebook domain, thousands of their invisible tentacles for data collection are reaching almost half of the world wide web. Our research on Online Trackers revealed that on the top 50 websites in Serbia that we use there are in 46% of the cases some of the Facebook cookiesembed.

“Cookies are small pieces of text used to store information on web browsers. Cookies are used to store and receive identifiers and other information on computers, phones, and other devices. Other technologies, including data we store on your web browser or device, identifiers associated with your device, and other software, are used for similar purposes. We use cookies if you have a Facebook account, use the Facebook Services, including our website and apps (whether or not you are registered or logged in), or visit other websites and apps that use the Facebook Services (including the Like button or our advertising tools).” 
Facebook Cookies Policy

According to the research we conducted on the 50 most frequently visited websites from Serbia there are in average 7 different 3rd party cookies embedded in every website we examined. In total, we detected 174 different types of cookies detected 365 times. Those 174 unique cookies belongs to 87 different companies.  There is massive dominance of 4 big US companies: Google (90%), Facebook (46%), Twitter (24%) and Amazon (10%) as well as the Infomediaries Gemius SA (36%), Httpool (7%).

Every time we visit some of those websites, Facebook receives information about our visit and this information is becomes an integral part of the profiling process, a never ending process of creating a clear picture about who you are, what you like and what are your behavioural patterns.

Mobile phone permissions

Even with the use of cookies Facebook is able to get information about your online behaviour, this is just a little part of the information compared to what they can get from your smart mobile devices.
According to our previous research Invisible Infrastructures : Mobile permissions , by installing Facebook, Facebook messenger, WhatsApp and Instagram you are giving access and right to exploit vast amount of different types of data stored on your phone. Some of those permissions are allowing Facebook to extract different forms of information that can be really intrusive such as device identifier, precise location of your device, identity of your contacts, content of your SMS messages, Your call log, record audio, get information about Your WiFi connection, download files without notification and many more.

Other Facebook Companies

At the time of our research, except for the main company, Facebook owned and operated 7 other companies : Facebook Payments Inc., Atlas, Instagram LLC, Onavo, Parse, Moves, Oculus, LiveRail, WhatsApp Inc. and Masquerade. According to them, they may share information about you within their family of companies to facilitate, support and integrate their activities and improve their services. Some of those “family” members are data collector giants as well. WhatsApp alone had  over 1 billion monthly active users worldwide as of February 2016. In June 2016 Instagram, another Facebook family member, had reach 500 million monthly active users. Those apps, especially WhatsUp are collecting even more information about user behaviour and activities. Ev en a brief look at the WhatsApp privacy policy or a list of mobile phone permissions, reveals a data collection operation similar in scale to the one that we are investigating within Facebook itself. Specifics to the services that some of those companies provide, the field of data collection is expanded to new frontiers. In the case of Oculus Rift, according to their privacy policy, they can collect and provide Facebook with information about your physical movements and dimensions when you use a virtual reality headset.  Facebook Payments Inc. is a company that provides  payment services on Facebook, while collecting different set of information, mostly related to your transactions, credit card numbers etc.

Detailed investigation of all the type of data that Facebook can collect through their companies other than Facebook itself will require extended analysis that we will unfortunately have to leave for some other investigation in the future.

Facebook partners

According to FB privacy policy they receive information about you and your activities on and off Facebook from third-party partners, such as information from a partner when they offer joint services or from an advertiser about your experiences or interactions with them.
In April 2013, Facebook  launched “partner categories” and incorporated offline and third-party data from data brokers  Acxiom (enterprise data and analytics), Datalogix (a digital media and offline purchasing data service), and Epsilon (direct-to-consumer marketing) to all categories of Facebook advertising. According to the New York Times article “Mapping, and Sharing, the Consumer Genome” from 2012, Axciom Corporation servers process more than 50 trillion data “transactions” a year. Company executives have said its database contains information about 500 million active consumers worldwide, with about 1,500 data points per person. That includes a majority of adults in the United States.

These companies collect information about you through things like store loyalty cards, mailing lists, public records information (including home or car ownership), browser cookies, and more. So, if you are buying at Safeway, and use your Safeway loyalty card that information is collected and saved by another Facebook partner company – Datalogix. In December 2014, Oracle Corp. had acquired Datalogix for $1.2 billion. According to their statement, Datalogix aggregates and provides insights on over $2 trillion in consumer spending and have over 650 customers including the top US advertisers and digital media publishers. According to them, with Datalogix, Oracle Data Cloud will deliver the richest understanding of consumers across both digital and traditional channels based on what they do, what they say, and what they buy enabling leading brands to personalise and measure every customer interaction.

Except with the biggest data collectors and dealers at the market, Facebook is exchanging data with hundreds of other data dealers, Ad technology developers, data and marketing analysis companies through their Facebook Partners program.

Another group of organisations that have access to Facebook data are vendors, service providers and other partners that are providing technical infrastructure services, analysing how our Facebook services are used, measuring the effectiveness of ads and services, providing customer service, facilitating payments, or conducting academic research and surveys. Facebook claims that these partners must adhere to strict confidentiality obligations.

In this part of our story we explored different forms and methods of data collection, massive operation hidden behind screens, code, embedded in pixels and cookies, performed by our devices and orchestrated by Facebook.

How this huge amount of data flows further, and how it is used, we will investigate in the next chapter of our story: Facebook Algorithmic Factory (2) : Human Data Banks and Algorithmic Labour

SHARE LAB 2016

Vladan Joler – Research, text, data collecting and visualization
Andrej Petrovski – Research, text and proofreading
Contributors : Kristian Lukic and Jan Krasni

1. Michael Evans, Karl Marx, London, England, 1975. Part II, Chap. 2, sect. a; p. 63
2. “Free Labor: Producing Culture for the Digital Economy”