The scientific product of my PhD Research.

For my PhD, I studied the impact of disinformation in information systems aimed to reach a consensus regarding a future recommendation or, possibly, a future decision.

Consensus from a crowd

As weird as it seems, crowds are very good at precise estimation when there is low possibility to mutually influence the estimation response from individuals. This is something already studied more than one centurt ago by Galton and that lately has been rebranded as the “wisdom of the crowd”.

So a crowd-rating system collect the opinions from a convenient sample of supposedly informed people and combine them into statistics for evaluation or decision making.

How disinformation distorts consensus

“Supposedly” is the keyword here: since the sample is not randomised and people can observe past statistics about the systems of its elements, disinformation can be enacted in two ways:

Tactical disinformation

  • My response to topic X is A
  • but I have incentives to make it recorded that my response is B.
  • B is someway different than A.
  • Possibly, converging into B will avoid that consensus on X will be C.

This is formally identical to tactical voting in political theory, where people would desire A to be elected, but they choose B because they do not believe that A would be elected, and they fear that C would, instead.


Sock-puppets

Sock-puppets are online fake identities, or just ‘fake accounts’.

  • I am Aleph and everybody think that I believe C regarding X.
  • I will make the fake online identity of Beth;
  • everybody will think that she believes C, too.
  • Conclusion: now the contribution of my opinion on X will count for double.

Here one can notice that Fake identities could induce a shift of response (not opinion, just response or “voting”) in tactical voters, inducing a false, disinformed, perception of future scenarios.

Social Botnetsare a extreme case of disinformation through sock-puppets.


In both cases, the disinformation do not seem per se primary aimed to conceal the latent consensus regarding the topic. Think about it: Aleph do not believe at all that he is telling a lie while trying to make other people think that A is the consensus. The lie consist in a false identity.

Suggested readings

The case for disinformation as a conceal of information about ourself - not about the outside world - is well discussed in Bergstrom & West’s Calling Bullshit. This theory, originally proposed by philosopher Harry Frankfurt, was further developed by authors as:


Mutully antagonistic acts of disiformation

If you accepted that Disinformation in consensus systems can go two-way, now you can imagine that sock-puppets can be antagonized by tactical disinformation!

  • I noticed that Aleph has a sock-puppet, Beth.
  • Aleph is pushing his agenda to promote opinion C on topic X!
  • Hence, I am morally justified to shift my opinion from A to B, in order to contrast opinion C!

I documented this dynamic in action in a relevant case of so-called Review Bomb, in a Online Review Platform (ORP). ORPs are a natural case to characterize as a consensus systems: each user can express to each item one opinion, formalized into a integer in a scale.

Methodology for reduction of disinformation impact

I believe that the most interesting contribution of my research concerns the development of the ABNM - MC methodology to support applications to reduce the impact of disinformation in consensus systems.

Agent Based Network Models (ABNM)

The best way to describe Agent Based Networks is:

…they are just randomly generated dynamic Networks where there is at least one class of nodes that are Agents.

Hyperparameters of the nodes and Agent behaviour are coded following a Model. By the actions of agents, the network will evolve into peculiar topologies that can be summarised with statistical indicators of outcome.

I think that a good ABNM accounts for agents observing past statistical summarisation of the whole systems, and acting accordingly in the present. Agent Based Network Models are developed at Indiana University - CNetS:NaN

Why is the word Network so important to define a subclass of Agent Based Models (ABM)? Because networks are both flexible and precise for model identification. I think that the epistemology of networks is still developing, so I will just provide an example of why networks are so interesting to model (social) action:

Imagine a recommender systems that observes actions from the users; these are very heterogeneous agents that interact with items in the catalogue of the systems. The items have no direct agency, we will call this structure a bipartite directed network. However, the recommender system acts as a architecture, shaping the topology of the connections between users and items. Even if not explicit, this mechanism can be found in one of the foundational paper on Consensus/ORPs systems.

My personal insight on this is that for facilitate and possibly enrich statistical inference, networks can always be converted into a tabular format, for example with softwares like tidygraph. Statistically speaking, that format would be a structured dataset - that means that one can apply Multilevel Statistics on (bipartite) networks.

(…Even if probably not exactly the classical estimators, since the assumptions are different…)


Monte Carlo (MC)

In the other panel it is said that agents doing stuff in a network lead into stable topologies that can be represented through statistical indicators.

There is an issue: while the class of the topology could be classified as unique, that means that all of randomly generated observation from the ANBM will share the same unique and known hyper-parameterisation, the statistical outcomes are still random variables with a unknow a priori variance.

Monte Carlo (MC), that is random resampling of the model would help identification of location of statistical indicators of outcome of the model. I personally find MC helpful for identification of the condition of neutrality of the model (null hypothesis), too.


Assumptions: why do we need an artificial simulation for understanding how to reduce disinformation?

Short answer: because in vivo experimentation on how real systems react to disinformation is straightforward unethical.

Long answer: not only that. If you read the panel about ANBM - MC and you know a bit of Econometric Theory, maybe you recognize something: the real world is just one observation, but for a theory of intervention to reduce the impact of disinformation, one have to infer potential outcomes. Here the potential is also tied to the level of disinformation in the system.

We already know that the quantity of disinformation in a system can be measured as the frequency of agents of disinformation in that systems or as the frequency of acts of disinformation, but the final misinformation (or just: bias) in the statistical indicators of consensus systems, while being dependent by the disinformation in the input, is not (necessarily) linearly dependent.

Just with real world data, this is hard to infer. Instead, real world data can be studied to fit more general hypothesis driven by scientific theories and theoretically backed statistical models. These general prior knowledge on models is then combined with random noise to Monte Carlo generate a posterior distribution of outcome statistics. The trick is that after identification of the disinformation-neutral (0 impact) model, one can raise the level of disinformation and map the relationship between the level of disinformation and the outcome statistics.

A Final Summary of the Methodology

ANBM-MC for estimation of the reduction impact of disinformation in consensus systems can be summarized in three steps:

First step: Data collection and analysis

  1. A context for consensus disinformation is identified
  2. Data is collected (possibly, scraped)
  3. Data is analysed to understand basic agent mechanics.
    • Possibly, disinformation is disentangled from genuine, sincere, opinions from the agents. So, on pair of mechanics to characterise honest agents, different mechanics would characterise disinformation agents.
    • If all the agents have potential for disinformation, then disinformation is in the act, not in the agent.

Second step: Model generation, performance and validation

  • Models run through instances (parallel spaces, also called universes or worlds) and iteration (time).
    1. Fixed hyperparameters of the simulation, parameters for the instance are randomly generated. For example, the hyperparameter could be “The prior probability for a user to be a tactical voter is 30%”. But since the istance is random, the final parameter for that statistics would be: “There are 32 tactical voters for each 100 users”.
    2. Then, first iteration starts.
    3. All agents execute their program.
    4. Summary statistics of the state of the system are collected and recorded as the current state of the system.
    5. Second iteration starts. It is possible that the behaviour of the agents will be influenced by the summary statistics at the end of the first instance.
    6. The iterations repeat until the final iteration, which could be pre-fixed (as a hyperparameters) or until the state of the system is steady.
    7. The it starts the second universe. Universes could even run in parallel since they are statistically independent.
    8. For each universe, final summary statistics at the last iteration are collected and recorded as outcomes in the table for results.
  • Through hyperparameterisation of the model, it is possible to explore many different setups. For example, the % of imputed disinformation could range uniformly from 0% to 30%, but never exceed 30%.
    • Given one of more target parameters per universe, do a raise in disinformation induce a stochastic raise in the bias for the statistics of a
    • For example: imagine that agents are programmed to infer the number of objects in a jar (which is a parameter for that universe). In the end, the average estimate across agents, that is a statistic, would be equal to that parameter plus one error. This error is tied to that universe in particular.
      • So… is the square of that error correlated with the % of disinformation in the system? This is a causal question that the ANBM - MC allows to answer to.
  • Validation of the ANBM - MC model: do the distribution of outcomes fit, or at least is it coherent in its projection over empirical data? And if there are strong differences, are these admissible or at least useful for the research? If not, the model is not valid to go further into step III.

Third Step: Estimation of the Counter-measure

There are two kind of counter-measures:

  • Endogenous interventions: which is likely a change of architecture within the consensus system.
    • Here is important that the intervention does not change the way that agents are programmed or parameterised. Only the condition to access about summary information about the sate of the systems. For example, this information could be totally negated to the agents, or presented in a different way, or only to some agents and not for others.
    • In this case the summary statistics for the table of results are exactly the same. The difference would be that in half of the universes, the intervention is applied, and in the other half is not. If the bias, that is the difference between parameter and consensus-reached estimate is lower in the sample with intervention, then the intervention for reduction of impact of disinformation has been estimed successful!
  • Exogenous interventions: this is trickier but it can be summarised in a very simple sentence: are there alternative statistics? i.e. are there alternative inferential models that out-perform the null statistics for consensus estimation in a disinformed system? Here the estimation method is just a bit more complicated, and for once, equations make it simplier:

$$ DIR = \tau(\theta_1(\hat{\alpha}),\alpha) - \tau(\theta_0(\hat{\alpha}),\alpha) $$

  • α is a vector of parameters.

    • For example: the latent quality of items in a ORP
  • θ are the estimators.

    • For example: the average of the rating received (theta zero) vs. the average of the rating received by the 100 most active user nodes (theta one)
  • τ is a coefficient of rank-correlation.

    In other words, for a exogenous change of estimator it only matters that it is robust to disinformation in order to be more desirable.


Posted on:
January 1, 0001
Length:
10 minute read, 1981 words
See Also: