Miguel Torres Costa

Wordle

2022-01-18T00:00:00+00:00

This is a solver for the Wordle game. Write the word suggested, type back the resulting colours as a string (i.e. BBYGG means the first two tiles are black, the middle is Yellow and the last two are Green), and this will suggest you the words needed to solve the game.

A newer version of this solver is at tools.mptc.uk/wordle.

Loading Component

Easy to State, Hard to Solve

2021-07-01T00:00:00+00:00

Interesting problems tend to follow a pattern of being very easy to state, and hard to solve. I am sharing one as an example. Do try to solve it before reading through the solution. Enjoy!

Problem

In how many ways can you cover a $ 3 \times n $ grid with $ 2 \times 1 $ sized dominoes?

Solution

Approach

We will write the number of solutions recursively, and deduce the general formula based on this.

Recursions

After adding dominoes we get grids that might not be $ 3 \times n $, so let us define these three values:

$A_n$ : The number of coverings of a $ 3 \times n $ grid with an extra cell in the $n+1$ th column.
$B_n$ : The number of coverings of a $ 3 \times n $ grid with two extra adjacent cells in the $n+1$ th column.
$C_n$ : The number of coverings of a $ 3 \times (n+1) $ grid.

With these in mind, we get three key recursions:

\[\left\{\begin{matrix} A_n = & B_{n-1}\\ B_n = & C_{n-1} + A_{n-1}\\ C_n = & B_{n-1} + C_{n-2} + A_{n} \end{matrix}\right.\]

Using $A_n = B_{n-1}$ to get rid of all the $A_n$ terms we get the system of equations

\[\left\{\begin{matrix} B_n = & C_{n-1} + B_{n-2}\\ C_n = & 2 B_{n-1} + C_{n-2} \end{matrix}\right.\]

In order to get similar recursions, we can define $C’_n = \sqrt{2} C_n$ to get the system of equations.

\[\left\{\begin{matrix} B_n = & \sqrt{2}C'_{n-1} + B_{n-2}\\ C'_n = & \sqrt{2} B_{n-1} + C'_{n-2} \end{matrix}\right.\]

Since $B_n$ and $C’_n$ look really similar, we can try to work out this system in terms of their average value and difference, by setting

\[\left\{\begin{matrix} \alpha_n = & \frac{B_n + C'_n}{2}\\ \beta_n = & \frac{B_n - C'_n}{2} \end{matrix}\right. \iff \left\{\begin{matrix} B_n = & \alpha_n + \beta_n\\ C'_n = & \alpha_n - \beta_n \end{matrix}\right.\]

Substituting back in the original equations we get

\[\left\{\begin{matrix} B_n = & \sqrt{2}C'_{n-1} + B_{n-2}\\ C'_n = & \sqrt{2} B_{n-1} + C'_{n-2} \end{matrix}\right. \iff \left\{\begin{matrix} \alpha_n + \beta_n = & \sqrt{2} \left ( \alpha_{n-1} - \beta_{n-1} \right ) + \left ( \alpha_{n-2} + \beta_{n-2} \right ) \\ \alpha_n - \beta_n = & \sqrt{2} \left ( \alpha_{n-1} + \beta_{n-1} \right ) + \left ( \alpha_{n-2} - \beta_{n-2} \right ) \\ \end{matrix}\right. \iff \left\{\begin{matrix} \alpha_n = & \sqrt{2} \alpha_{n-1} + \alpha_{n-2} \\ \beta_n = - & \sqrt{2} \beta_{n-1} + \beta_{n-2} \\ \end{matrix}\right.\]

These 2 equations are independent of each other and relatively simple to solve :)

$\alpha_n$: Solving $ \alpha^2 = \sqrt{2} \alpha + 1 $ we get $ \alpha = \frac{1 \pm \sqrt{3}}{\sqrt{2}}$, which leads to the general solution $ \alpha_n = r \left ( \frac{1 + \sqrt{3}}{\sqrt{2}} \right )^n + s \left ( \frac{1 - \sqrt{3}}{\sqrt{2}} \right ) ^n$.
$\beta_n$: Solving $ \beta^2 = - \sqrt{2} \beta + 1 $ we get $ \beta = \frac{-1 \pm \sqrt{3}}{\sqrt{2}}$, which leads to the general solution $ \beta_n = t \left ( \frac{-1 + \sqrt{3}}{\sqrt{2}} \right )^n + u \left ( \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) ^n$.

With these general formulas, we can get

\[\left\{\begin{matrix} B_n & = & B_n & = & \alpha_n + \beta_n & = & r \left ( \frac{1 + \sqrt{3}}{\sqrt{2}} \right )^n + s \left ( \frac{1 - \sqrt{3}}{\sqrt{2}} \right ) ^n + t \left ( \frac{-1 + \sqrt{3}}{\sqrt{2}} \right )^n + u \left ( \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) ^n\\ C_n & = & \sqrt{2}C'_n & = & \sqrt{2} \left ( \alpha_n - \beta_n \right ) & = & \sqrt{2} \left ( r \left ( \frac{1 + \sqrt{3}}{\sqrt{2}} \right )^n + s \left ( \frac{1 - \sqrt{3}}{\sqrt{2}} \right ) ^n - t \left ( \frac{-1 + \sqrt{3}}{\sqrt{2}} \right )^n - u \left ( \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) ^n \right ) \end{matrix}\right.\]

The final step is to find the values of $r$, $s$, $t$ and $u$. For that we need to figure out basecases, namely:

\[\left\{\begin{matrix} B_0 & = & 1\\ C_0 & = & 0\\ B_1 & = & 1\\ C_1 & = & 3 \end{matrix}\right.\] \[\left\{\begin{matrix} r+s+t+u & = & 1\\ \sqrt{2} \left ( r+s-t-u \right ) & = & 0\\ r \frac{1 + \sqrt{3}}{\sqrt{2}} + s \frac{1 - \sqrt{3}}{\sqrt{2}} + t \frac{-1 + \sqrt{3}}{\sqrt{2}} + u \frac{-1 - \sqrt{3}}{\sqrt{2}} & = & 1\\ \sqrt{2} \left ( r \frac{1 + \sqrt{3}}{\sqrt{2}} + s \frac{1 - \sqrt{3}}{\sqrt{2}} - t \frac{-1 + \sqrt{3}}{\sqrt{2}} - u \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) & = & 3 \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ (r+s) \frac{1}{\sqrt{2}} + (r-s) \frac{\sqrt{3}}{\sqrt{2}} + (t+u) \frac{-1}{\sqrt{2}} + (t-u) \frac{\sqrt{3}}{\sqrt{2}} & = & 1\\ (r+s) - (r-s) \sqrt{3} + (t+u) - (t-u) \sqrt{3} & = & 3\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ \frac{1}{2} \frac{1}{\sqrt{2}} + (r-s) \frac{\sqrt{3}}{\sqrt{2}} + \frac{1}{2} \frac{-1}{\sqrt{2}} + (t-u) \frac{\sqrt{3}}{\sqrt{2}} & = & 1\\ 1 + ((r-s)-(t-u)) \sqrt{3} & = & 3\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ (r-s) \frac{\sqrt{3}}{\sqrt{2}} + (t-u) \frac{\sqrt{3}}{\sqrt{2}} & = & 1\\ ((r-s)-(t-u)) \sqrt{3} & = & 2\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ (r-s) \frac{\sqrt{3}}{\sqrt{2}} + (t-u) \frac{\sqrt{3}}{\sqrt{2}} & = & 1\\ (r-s)-(t-u) & = & \frac{2}{\sqrt{3}}\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ (r-s) + (t-u) & = & \frac{\sqrt{2}}{\sqrt{3}}\\ (r-s)-(t-u) & = & \frac{2}{\sqrt{3}}\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r+s & = & \frac{1}{2}\\ t+u & = & \frac{1}{2}\\ r-s & = & \frac{1}{\sqrt{6}} + \frac{1}{\sqrt{3}}\\ t-u & = & \frac{1}{\sqrt{6}} - \frac{1}{\sqrt{3}}\\ \end{matrix}\right.\] \[\left\{\begin{matrix} r & = & \frac{1}{4} + \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}}\\ s & = & \frac{1}{4} - \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}}\\ t & = & \frac{1}{4} + \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}}\\ u & = & \frac{1}{4} - \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}}\\ \end{matrix}\right.\]

Assembling all of this together we get that $C_n$ equals the sum of the four terms

\[C_n = \sqrt{2} \left ( \left ( \frac{1}{4} + \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{1 + \sqrt{3}}{\sqrt{2}} \right )^n \\ + \sqrt{2} \left ( \frac{1}{4} - \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{1 - \sqrt{3}}{\sqrt{2}} \right ) ^n \\ - \sqrt{2} \left ( \frac{1}{4} + \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{-1 + \sqrt{3}}{\sqrt{2}} \right )^n \\ - \sqrt{2} \left ( \frac{1}{4} - \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) ^n \right )\]

Since $C_n$ is the number of solutions of a $3 \times (n+1) $ grid, our answer is

Answer

\[\sqrt{2} \left ( \left ( \frac{1}{4} + \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{1 + \sqrt{3}}{\sqrt{2}} \right )^{n-1} \\ + \left ( \frac{1}{4} - \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{1 - \sqrt{3}}{\sqrt{2}} \right ) ^{n-1} \\ - \left ( \frac{1}{4} + \frac{1}{2 \sqrt{6}} - \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{-1 + \sqrt{3}}{\sqrt{2}} \right )^{n-1} \\ - \left ( \frac{1}{4} - \frac{1}{2 \sqrt{6}} + \frac{1}{ 2 \sqrt{3}} \right ) \left ( \frac{-1 - \sqrt{3}}{\sqrt{2}} \right ) ^{n-1} \right )\]

Mathsy Shortcuts

2021-04-18T00:00:00+00:00

I have written this page to add small shortcuts and insights in reasoning that can be useful.

Geometric mean is the (arithmetic) mean of magnitudes

Geometric mean is the (arithmetic) mean of magnitudes

For a set of positive numbers $ A = \{ x_1, x_2, … ,x_n \} $ the arithmetic mean is defined as $\frac{1}{n} \sum {x_i} $ whereas the geometric mean is defined as $ \sqrt[n]{ \prod x_i} $, resulting in:

$ A $	Arithmetic Mean	Geometric Mean
$ \{ 4,9 \} $	$6.5$	$ 6 $
$ \{ 1,2,4,8,16 \} $	$6.2$	$ 4 $

The Geometric Mean becomes obvious if we write all numbers as magnitudes:

$ A $	Arithmetic Mean	Geometric Mean
$ \{ 2^2,2^{3.1699} \} $	$ 2^{2.7004} $	$ 2^{2.58496} $
$ \{ 2^0,2^1,2^2,2^3,2^4 \} $	$ 2^{2.632268} $	$ 2^2 $

Note that $2.58496$ is just the mean of $ \{ 2 , 3.1699 \}$, and for the second line $2$ is the mean of $ \{ 0,1,2,3,4 \}$. I have written them base $2$ but any magnitude would suffice.

KPIs - Lag vs Lead Measures

2021-04-18T00:00:00+00:00

KPIs have proven themselves as a way to ensure we stay on track and actually achieve our goals, be it at a business level (i.e. How many new clients are we getting each day?) or a personal level ( i.e. how many hours of exercise am I doing each week? ). As I read through Cal Newport’s Deep Work he distinguishes between lead and lag measures, a big enough distinction that warrants keeping in mind while designing any KPI.

Lag measures are more in line with what we want to achieve ( i.e. business profitability or exam scores ), but they are only observed after the correct decisions are made. This lag makes them less useful for driving decisions, as they only inform us on the quality of past decisions.

Lead measures are usually less aligned with what we care about, but provide us with earlier information about what we care about, even if it isn’t always perfect information.

In order to practice building good lag and lead measures, I have started building a few examples. In all these examples, the Lagged Measures are what I mostly care about, but the Lead Measures allow me to consistently build towards that goal, even if they don’t perfectly reflect it.

Goal	Lagged Measure	Lead Measure
Becoming healthier	Overall tiredness during the day	Quality of the food I am eating
Better grades	Exam scores	Time it takes to correctly complete homework
Sucessfully launching a startup	Monthly revenue	Number of new clients every week
Social welfare	Population satisfaction	Wealth gini coefficient
Financial security	Passive income	Active income

Which other examples should I add?

Strange Attractors

2021-03-24T00:00:00+00:00

This is an implementation of Numberphile’s ideas from their Chaos Game Video. Click on Run Simulation below to see the magic happen!

Intro
Tool
Can you replicate these patterns?

Intro

In the plot you will see the latest point (orange) and a set of attractors. At every iteration, a random attractor is selected and the latest point becomes the midpoint between the attractor and the orange point. Repeat this thousands of times and fractals start to emerge.

You can play around with the list of attractors, number of iterations per simulation and the strength of the pull towards the attractor.

Tool

Loading Component

Can you replicate these patterns?

Sierpinski Triangle

Sierpinski Tetrahedron

Sudoku

Big Sudoku

Octogon

Cursor

Falcon

Perspective

Tic Tac Toe

Trapezoid

All horses have the same colour

2021-03-20T00:00:00+00:00

Mathematical Induction has 3 steps to it:

Induction Hypothesis: state the identity that we want to prove.
Base Case: prove that the property is true for $n=1$.
Inductive Step: show that if the property is true for $n$, then it must be true for $n+1$.

This works because the base case shows the property holds for $n=1$. Then we apply the inductive step to show that it holds for $n=2$, which implies that it holds for $n=3$, and so forth for all positive integers. As such, I will show the following statement:

All horses have the same colour pattern

Induction Hypothesis: Any set of $n$ horses $ h_1, h_2,… h_n $ have the same colour pattern.
Base Case: For $n=1$ we have a single horse, so there can be only one colour pattern.
Inductive Step: Assume that the statement is true for $n$. Take the set $h_1, h_2,…, h_n, h_{n+1} $ with $n+1$ horses. By our hypothesis the set of horses $h_1,…,h_n$ has a single colour pattern, and the set $h_2, … , h_n, h_{n+1} $ also has a single colour pattern. Since this must be the colour pattern of their intersection set $h_2,…h_n$, then the entire set of $n+1$ horses has the same colour.

All three elements of mathematical induction are fulfilled. Can you spot the logical error?

MasterMind - Game

2021-03-10T00:00:00+00:00

Goal

This is a rendition of the old game Master Mind. Specify the length of the alphabet and the number of hidden characters to get started. When submitting a guess, you will get crosses X for every correct guess in the correct position, zeroes O for correct guesses in the wrong position, and dashes - for the remaining guesses.

Best of luck!

Loading Component

Asset Division

2021-03-06T00:00:00+00:00

Goal

During inheritance procedures or other events of distribution of shared wealth, finding a satisfactory distribution can be challenging. Below we try to solve the issue based uniquely on the people involved and a list of items to be distributed, assuming that all have equal rights over the shared items.

A newer version of this tool is at tools.mptc.uk/split.

Technical Note

Assuming that each person has a dollar value for each item, not necessarily equal to the other members, the fact that there is a distribution accepted by everyone is assured by Sperner’s Lemma. This does not take into account possible psychological factors, such as only accepting a distribution if the other parties are not happy with it. Instead it assumes a pure profit maximizing approach.

Steps

Type in the list of people wanting to divide the items.
Add the items, and for each item an estimated price. This does not need to be exact nor correct as the algorithm re estimates the prices, it just helps to initialize it with sensible values.
Click on “Start” to freeze the above lists and start estimating.
Each round you will be shown different groupings of the items, and each person should select their favorite group each round. This will allow us to infer item valuations, and suggest the best item allocation.

Loading Component

Integrating Factors

2021-01-17T00:00:00+00:00

Integrating factors allow us to solve equations of the form $y' + P(x)y = Q(x)$

Intuition

The derivative of $e^{R(x)}$ is $R'(x) e^{R(x)}$ . This can be used to cancel out terms when we have anything of the form $y' + R'(x)y$, and we can do so by setting $P(x) = R'(x)$ . By playing around with this idea the methodology below was developed.

General Solution

We know that $y' + P(x)y = Q(x)$. We want $R'(x) = P(x)$, so we define $R(x) = \int P(x) dx$ to get $y' + R'(x)y = Q(x)$. By multiplying all terms of the equation by $e^{R(x)}$ we get

\[\begin{align} Q(x) e^{R(x)} &= y'e^{R(x)} + yR'(x)e^{R(x)} \\ &= y'e^{R(x)} + y(e^{R(x)})' \\ &= (ye^{R(x)})' \end{align}\]

By integrating both sides and then rearranging for $y$, we get $ye^{R(x)} = \int Q(x) e^{R(x)} dx \Leftrightarrow y = e^{-R(x)} \int Q(x) e^{R(x)}dx$, with $R(x) = \int P(x) dx$ as initially defined.

Example

Many examples require harder integrals than the one that follows, however this example should give you an idea of how to proceed.

\[y' - \frac{y}{x} = x\]

We have that $P(x) = -\frac{1}{x}$, so $R(x) = \int P(x) dx = - \int \frac{1}{x} dx = -\ln{x}$. Hence our Integrating Factor is $e^{-\ln{x}} = \frac{1}{x}$, so we can rewrite our original equation by multiplying it by $\frac{1}{x}$ as

\[\begin{align} \frac{y'}{x} - \frac{y}{x}\frac{1}{x} &= x \frac{1}{x} \\ \frac{y'}{x} - \frac{y}{x^2} &= 1 \\ \frac{y'}{x} - y \left ( \frac{1}{x} \right ) ' &= 1 \\ \left ( \frac{y}{x} \right ) ' &= 1 \end{align}\]

Integrating both sides with respect to $x$ we get

\[\frac{y}{x} = \int 1 dx = x \Leftrightarrow y = x^2\]

Confirming it within the original equation, we indeed get $y' - \frac{y}{x} = ( x^2 ) ' - \frac{x^2}{x} = 2x - x = x$

Priority setting - should we train guide dogs for the blind?

2020-10-23T00:00:00+00:00

This is the first of (what I hope to be) many projects I will write about here. They are better based on tools that I think would be useful for society as a whole rather than profitable.

Should we train guide dogs for the blind? The typical answer is yes, but I have a strong reason against it: you are preventing thousands of other people from recovering from blindness by doing so. The reason for this claim is as follows: The entire process of training dogs can cost over $ 40000 USD. For the same price, we could treat thousands with Anti Vascular Endothelial Growth Factor Therapy. So my answer is clear: I am strongly against training Guide Dogs.

Whereas the case above might not hold ( I am sorry doctors if I mentioned something clearly wrong, do feel free to correct me ), I hope the broader point is clear: many movements and decisions happen at a societal level due to the awareness and emotion that they create, rather than the actual benefit that they provide to humanity. In order to align society wide benefits with popular sentiment, I am keen on building a tool that allows for direct comparison of actions against their societal benefit AND the cost of implementing them, in practice reaping the benefits of a benefit cost analysis in a simple interface.

After some consideration, this is the structure that I believe would work best. Feel free to think this through yourself before reading my beliefs, I would be interested to compare them.

Key ideas

I want to show in a simple plot a comparison of the effectiveness of different actions at a society level. I could visually share a lot of information by using a 2d plot where you could hover and click to get more information about the specific measures.

Dimentions available to show information.

x-axis
y-axis
colours (rgb could potentially provide 3 dimentions. This could prove particularly useful for person clustering, in terms of understanding which groups favor which measures.
transparency
size

By a mix of the above I should be able to show up to 7 dimentions of data.

Possible information shared

What measures have the most positive impact on humanity?
Which measures are the most likely to happen?
How long ago was this measure proposed?
How many people are devoted / interested in this measure?
Many more things I haven’t though of but you could tell me :P

The main challenge is to actually gather the information about all measures. One possible way to do it is to build a platform.

Rough action steps

People can sign up to the platform and submit measures.
A full list of the measures would be presented, which people can favorite.
Favorited measures can be ranked on the above questions (ranking is much easier that building specific numbers).
Use the many partial orderings provided by the users into a total ordering.
Plot this information for users to in a single plot.
Submitted measures could make use of hashtags in order to allow for groupings. I would use #Portugal for measures about my home country, #Maths for measures on speeding up academic research (maybe which techniques to develop), and any others of interest.
As time passes questions could fade into the background.
Live happily ever after.

The final objective would be to end up in a situation where only Pareto Efficient measures end up being applied, in the sense that for any measure, if there is an alternative which is simultaneously better and easier to implement, then we should not waste time on it. Interested? Let’s make this happen :P

Shopping Cart

2020-10-08T00:00:00+00:00

Found on the web

The shopping cart is the ultimate litmus test for whether a person is capable of self-governing.

To return the shopping cart is an easy, convenient task and one which we all recognize as the correct , appropriate thing to do. To return the shopping cart is objectively right. There are no situations other than dire emergencies in which a person is not able to return their cart. Simultaneously, it is not illegal to abandon your shopping cart. Therefore the shopping cart presents itself as the apex example of whether a person will do what is right without being forced to do it. No one will punish you for not returning the shopping cart, no one will fine you or kill you for not returning the shopping cart. You must return the shopping cart out of the goodness of your own heart. You must return the shopping cart because it is the right thing to do. Because it is correct.

A person who is unable to do this is no better than an animal, an absolute savage who can only be made to do what is right by threatening them with a law and the force that stands behind it.

The Shopping Cart is what determines whether a person is a good or bad member of society.

Pricing Information

2020-09-10T00:00:00+00:00

Expected Opportunity Loss is a simple yet powerful way to price information, first introduced to me in Douglas’ Hubbard’s book “How To Measure Anything”

After reading it, the concept is simple: In many situations we have to make decisions under uncertainty, and as such after the fact we will discover that we could have made a better choice, that is, we missed an opportunity. By quantifying the likelihood of uncertain events, we can understand how much we expect to miss out and either make better decisions based on this value, or gather more information.

To start with an unrealistically simple event, you choose heads or tails, I flip a coin and if you got heads correct you receive $ 10€ $, if you got tails correct you receive $ 20€ $.

You have to choices:

You say Heads: There’s a 50% chance you get heads, which is optimal, so you lost out nothing, and a 50% chance you get tails, which would have made you $ 20€ $ richer, so your opportunity loss is $ .5 * (0) + .5 * (20) = 10 $
You say Tails: There’s a 50% chance you get tails, which is optimal, so you lost out nothing, and a 50% chance you get heads, which would have made you $ 10€ $ richer, so your opportunity loss is $ .5 * (0) + .5 * (10) = 5 $

Under a regret minimization system, this makes it clear that we want to choose Tails. This should have been obvious from before, but there is something we gained from this analysis. Under our optimal choice our expected loss is $ 5€ $, so if we paid any amount smaller than $ 5€ $ to know the coin in advance, we would expect higher returns than if we didn’t take that option. In other words, knowing the outcome of this event is worth $ 5€ $ to us: any value above that and we will be missing out on some opportunity. Any value below it and we can increase our expected profits.

The above is detached from reality, but instead of heads or tails suppose we are talking about presidential candidates getting elected, or investing in a new stock. Add a few zeroes to the values above and we will get a realistic price for getting information on the outcome, by doing voting polls or other mechanisms available to our situation.

This can be done for any decisions made under uncertainty (as Douglas Hubbard argues in his book).

I have written up this Information Pricing mechanism in my notes on Expected Opportunity Loss.

First Sign of Civilization

2020-09-08T00:00:00+00:00

Excerpt found on the web

Years ago, anthropologist Margaret Mead was asked by a student what she considered to be the first sign of civilization in a culture. The student expected Mead to talk about fishhooks or clay pots or grinding stones

But no. Mead said that the first sign of civilization in an ancient culture was a femur (thigh bone ) that had been broken and then healed. Mead explained that in the animal kingdom, if you break your leg, you die. You cannot run from danger, get to the river for a drink or hunt for food. You are meat for the prowling beasts. No animal survives a broken leg long enough for the bone to heal.

A broken femur that has healed is evidence that someone has taken time to stay with the one who fell, has bound up the wound, has carried the person to safety and has tended the person through recovery. Helping someone else through the difficulty is where civilization starts, Mead said.

We are at our best when we serve others. Be civilized.

Ira Byock

Perfect Forecasting

2020-08-19T00:00:00+00:00

The chapters below are an aggregation of thoughts around the theme of forecasting, not necessarily in a ready to present form, for a problem of joint decision making. We have society wide problems we need to agree on, such as which drugs should be illegal, which laws to pass, whether to implement an Universal Basic Income, and many other potentially controversial topics, and these topics are informed on forecasts that might turn out to be or not accurate.

We have played around with many

How can we aggregate multiple distinct forecasts into a single, more accurate, view.

Table of Contents
Assumptions for this quest
Event outcomes are not just true and false
We cannot observe the underlying probability of events, only the outcome
The value of better forecasts can be quantified
Existing companies in this area
The Future

Assumptions for this quest

The quest for perfect forecasting started a few years back, with a few assumptions from my side:

All future events can be assigned a likelihood of happening, between 0% and 100%
Due to different backgrounds and contexts, incompatible perspectives exist about the likelihood of many of these events.
The above can still be aggregated into a single “best” forecast using the correct system.

Event outcomes are not just true and false

Before we start talking about perfect forecasting, we should understand what problems we would like to forecast. These could be statements such as:

Facebook will go bankrupt before 2050
A baby will be born outside of Earth before 2100
A debt financial bubble will burst before 2025

These all seem rather trivial to check if they are true or not, however imagine the respective situations:

Facebook incorporates under a parent company, similar to Google and Alphabet, which acquires WhatsApp, Instagram and other Facebook owned companies. These surpass Facebook in size so Facebook eventually closes, but not the parent company. You could argue that technically Facebook closed, but you could also argue that the entire group just rebranded under the parent company.
Outer orbit flights become normal practice for long haul flights due to cheaper costs, and a baby is born in these flights. This is technically outside Earth’s atmosphere, but Earth is still the closest space body, making the outcome unclear.
A financial bubble bursts, but the origin is unclear.

From this there are 2 main action points I take in order to make:

Event outcomes always have a grey area, so they might be true, false or unclear. We need to take this into account when building forecasting metrics.
For better forecasting, it is worthwhile spending some time working out and removing the grey areas, by specifying terms and possible outcomes.

We cannot observe the underlying probability of events, only the outcome

Assume everyday I launch a die, and we forecast the statement

The die will show the number 6 up

For practical purposes this can’t be predicted with certainty. What we can do is still make the forecast that this statement has a $ \frac{1}{6} \approx 16.6 \% $ probability of being true. However after the event, we will either observe that the statement is true or not, we cannot observe the underlying probability.

As such overconfidence might be rewarded or punished based on chance. What we can ensure is that after enough of these events we can distinguish the best and worst forecasts by using reward functions that incentivize honesty. These are called Proper scoring rules.

The value of better forecasts can be quantified

The best book on this topic is How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard. Most of the book is spent on showing that all events that we care about can be measured, and convincingly so. A single chapter was dedicating to pricing information into a lot of detail. I have notes on Expected Opportunity Loss.

The key idea can be made clear by using an example. Assume that quantum computers might become a reality, and you can decide to invest on a startup in that area. For simplicity reasons, we will assume the 4 possible outcomes listed below, with full certainty of Return on Investment:

	Quantum computers are created before 2025	Quantum computers are not created before 2025
You invest in a Quantum computer company	You get 10 million EUR in return	You lose 1 million EUR (your initial investment)
You don’t invest in a quantum computer company	You neither lose nor get money	You neither lose nor get money

Further, the probability of Quantum Computers being created before 2025 is $p$.

If we invest in a Quantum Computer company, the Expected Loss is given by $p(0) + (1-p)(1) = (1-p)$ million EUR.
If we don’tinvest, the expected loss is given by $ p(10) + (1-p)(0) = 10p $ million USD

As such we should invest iff $p>\frac{1}{11}$, as can be seen by this table of Opportunity Losses in million EUR:

$p$	$0$	$\frac{1}{11}$	$\frac{1}{2}$	$1$
If we invest	$1€$	$\frac{10}{11}€$	$\frac{1}{2}€$	$0€$
If we don’t invest	$0€$	$\frac{10}{11}€$	$\frac{10}{2}€ = 5€$	$10€$

The key idea is that whenever a decision is made under uncertainty, you will lose money by making a non optimal decision. The Expected Opportunity Loss can be measured, and be reduced by reducing uncertainty around the unknown variables. As such say that currently $p=\frac{1}{2}$. By the optimal table the optimal decision is to invest, with an expected opportunity loss of half a million EUR. This means that for the possibilition to know now whether Quantum computers will be created before 2025 or not, any price below half a million EUR will still give us a profit. This is the price of perfect information for this unknown.

Note that this is a simplified example of a much more complex world: Not only do we have to take into account way more variables, we also often don’t have the possibility of fully reducing uncertainty, we can only get better measurements. Hubbard built a successful life around these measurements, and you can read more in my notes on Expected Opportunity Loss.

Existing companies in this area

At this point the opportunity in this area consists in building a platform for anyone to submit statements of interest for themselves, and other users to submit forecasts. With this we can rank users according to their forecasting abilities, and use this metrics to build smart aggregate forecasts.

Good Judgement Project

Arguably Good Judgement Project (gjp) are the first to the field, with the findings from their results being detailed in the book Superforecasting: The Art and Science of Prediction. The biggest shortcoming from gjp is that they filter all their questions for forecasting, and as such it is harder for users to submit questions they find interesting.

Foretold

The backend and flexibility of inputs on foretold is impressive, however they lack an intuitive User Interface. As such the existing user base has stagnated.

Metaculus

With the biggest user base, Metaculus has existed since 2015 and focuses on building questions with very well defined outcomes, covering many different categories. Since their approval process for new questions is incredibly thorough, they’ve built a backlog of non yet accepted questions that only promises to grow.

Augur

Most of the forecasting platforms above have to conform to laws in their areas. Augur does not for the same reason that Bitcoin does not: They are decentralized peer 2 peer networks. Instead of focusing on forecasts, Augur creates pair of coins for each statement, which trade independently of each other. However, one coin is only worth anything if the statement is true, and the other coin follows the reverse. As such these coins are traded on the basis of expected values, and we can extract forecasts from the market based on this idea.

Since no company controls Augur (similarly to how no company controls Bitcoin), then Augur does not need to worry about any issues with insider trading, assassination markets or other nefarious systems that arise if platforms are left unregulated.

Forecastapp aka Facebook

The latest entrant to the group, Facebook has launched forecastapp to crowdsource forecasts. Given their dominance in the social network world this makes it easier for them to gather forecasters, however their existing page on forecasting is still very simple so in beta testing, meaning that it is still unclear how the platform will evolve.

The Future

There’s an opportunity here to improve this platform. Can you do it?

The Hardest Logic Puzzle Ever

2020-07-25T00:00:00+00:00

I was recently given “The Hardest Logic Puzzle Ever”, which can be stated as follows:

Three gods A, B, and C are called, in no particular order, True, False, and Random. True always speaks truly, False always speaks falsely, but whether Random speaks truly or falsely is a completely random matter. Your task is to determine the identities of A, B, and C by asking three yes-no questions; each question must be put to exactly one god. The gods understand English, but will answer all questions in their own language, in which the words for yes and no are da and ja, in some order. You do not know which word means which.

Finding the solution

The first step in finding a solution is to know whether a solution exists or not. We start with having three goods, $(A,B,C)$, and we want to know which of the following triplets represents the gods:

$(True, False, Random)$.
$(True, Random, False)$.
$(False, True, Random)$.
$(False, Random, True)$.
$(Random, True, False)$.
$(Random, False, True)$.

So our $3$ questions need to be able to distinguish $6$ different possibilities. Since each question has $2$ possible responses, we would have $2*2*2=8$ possible outcomes.

But we can’t distinguish the words for yes and no. Doesn’t that reduce the answers we are able to distinguish in half, since if we swap all of the da’s and ja’s we would get the same information?

Not exactly. We are told what are the words for yes and no. Even though we don’t know which is which, we can use that information to get a yes/no answer even without knowing which is the word for yes and the word for no. As an example, suppose that I am asking whether the number $2$ is prime. Then I could ask: “If I ask is the number $2$ prime and does da mean no, would you give the same answer to both questions?”. Looking at the True/False gods and the possibilities da=yes / da=no, we get that their answers will be:

	True God, da=yes	True God, da=no	False God, da=yes	False God, da=yes
Is the number $2$ prime?	da (yes)	ja (yes)	ja (no)	da (no)
Does da mean no?	ja (no)	ja (yes)	da (yes)	da (no)
Would you answer both questions above equally?	ja (no)	ja (yes)	da (yes)	da (no)

We don’t have access to whether the given answer means yes or no, but from the above it is clear that the True God answers ja, and the False da, to a clearly true statement. If We swap the question “Is $2$ prime?” for a false statement such as “Is $4$ prime?”, both the 1st and 3rd rows would swap their da’s and ja’s, so the True God would now answer da and the False God would answer ja.

As such, by appending the last part “If I asked you if $P$ is true and if does da mean no, would you give the same answer?”, then I can act as if ja means yes and da means no.

Therefore we can skip the part about da/ja all together, and assume we get the usual English answers yes and no from our Gods.

Awesome, but what about the Random God?

The random God can indeed be a problem: The True / False Gods give us a lot of information (1 bit to be exact) whenever we ask a question, however The Random God introduces noise to the system. In particular when we ask the first question, because of the $1/3$ probability of there being a random God, we only get $1 \frac{1}{3} + 1 \frac{1}{3} + 0 \frac{1}{3} = \frac{2}{3}$ bits of information from our first question. As such instead of getting $3$ bits from $3$ questions, we only get $2+\frac{2}{3} \approx 2.66$ bits. This however is still larger than the $log_2(6) = 2.58$ bits of information needed to know which God is which, so it would still be possible to solve this question.

What do you mean bits of information?

This was a concept introduced by Claude Shannon in his thesis A Mathematical Theory of Communication. Make sure to check out his documentary The Bit Player if you want to learn more about the man behind the myth :) One of the key ideas introduced in that paper is a measure of information ( i.e. Entropy ) and can be measured in bits, which stands for binary digits. Each yes/no answer provides a bit of information, however that depends on the uncertainty of getting a yes/no answer. If we know that in Portugal it rains only 5% of the time, you telling me that it is not raining tomorrow does not give as much information as if there was a 50 / 50 chance of rain: Even without knowing the outcome I already could have made a very good guess to start with.

In the problem we are facing, we have $6$ options, and, as per Shannon’s work, this means we need $log_2(6) = 2.58$ bits of information. Each bit of information allows us to decrease the number of possible outcomes in half (or the ceiling of half), so with 1 bit we can go from $6$ options to $3$, another bit allows us to go to from $3$ to $\left \lceil 1.5 \right \rceil = 2$, and the last bit from $2$ to $1$ possibility (the final answer).

Alright, so how do we go from here to an answer?

Everything above was done to ensure we weren’t spending time on an impossible question, as many logic problems end up being. We need to ask $3$ questions that will filter our options to $1$. We need to ask an initial question which will give us 4 possible outcomes: the reason for that is that, looking at the possible orderings, there are $2$ in $6$ people that will answer truthfully, the same for lying, but then the $2$ options answered randomly can fall in any of the 2 cases above).

Let us start with asking the question “Is 2 prime?” to the first God. Since we know this to be true, we can distinguish True and False Gods, but not Random. As such if the answer is yes we get the possible orderings

$(True, False, Random)$.
$(True, Random, False)$.
$(Random, True, False)$.
$(Random, False, True)$.

where as if it false, we get the possible orderings

$(False, True, Random)$.
$(False, Random, True)$.
$(Random, True, False)$.
$(Random, False, True)$.

In both cases we get $4$ options. Since we have 2 questions left, we need to ensure that at each point we reduce the number of possibilities in half. There is a problem though: In every column above there is at least one row containing the Random God. The reason this is problematic is because we will not be able to distinguish the Random God from the most common God in that column, and as such there is always the possibility that we will end up with three options after getting the answer to our question.

What we need is a way of getting $4$ options, similar to the above, but Random God free columns. The only way to find such a question is to play around with possible questions to ask, and eventually I was able to find such a question: “Are you sitting in the cycle order True, False, Random?”, or in other words, are you in one of the three following orderings?

$(True, False, Random)$.
$(False, Random, True)$.
$(Random, True, False)$.

Let’s look at the different answers:

$(True, False, Random)$: The answer is yes, and since we ask a True God we get yes.
$(True, Random, False)$. The answer is no, and since we ask a True God we get no.
$(False, True, Random)$. The answer is no and since we ask a False God we get yes
$(False, Random, True)$. The answer is yes, and since we ask a False God we get no.
$(Random, True, False)$. Random God, so anything goes.
$(Random, False, True)$. Random God, so anything goes.

Hence, if we get the answer yes, the possibilities are

$(True, False, Random)$.
$(False, True, Random)$.
$(Random, True, False)$.
$(Random, False, True)$.

And if the answer is no, the possibilities are

$(True, Random, False)$.
$(False, Random, True)$.
$(Random, True, False)$.
$(Random, False, True)$.

We see that in the first case we can ask the 2nd God the remaining questions, knowing that they will not be random, and in the second case we can ask the 3rd God similarly. As such our job becomes much easier: We first ask the definitely non random God “Is 2 Prime?”” This will allows us to half the number of possibilities by telling us whether we are talking to the True or False God. Once we know that we will end up in one of the 4 possibilities:

Case A

$(False, True, Random)$.
$(Random, True, False)$.

Case B

$(True, False, Random)$.
$(Random, False, True)$.

Case C

$(False, Random, True)$.
$(Random, False, True)$.

Case D

$(True, Random, False)$.
$(Random, True, False)$.

All of those can be solved by asking the Gods we know with certainty whether the first God is random, and then we adjust the answer to the fact that we are talking to the True or False God.

So we’re done, hurray!

Pretty much. We skipped the da / ja instead of yes / no answers, so let’s just make our answer proper.

The answer

By using the trick “If I asked you if $P$ is true and if does da mean no, would you give the same answer?”, we can act as if da means yes for the statement “Is $P$ true?”.

We ask the first God: “If I asked if you are sitting in the cycle order True/False/Random and if da means no, would you give the same answer?”.

If the answer is da (yes), we end up with the certainty of a non random God in the 2nd position. Similarly, ja (no) gives us a definitely non random God in the third position.

Then we discover if the God is True or False by asking “If I asked if $2$ is prime and if da means no, would you give the same answer?”.

Lastly we know the other Gods by finalizing with “If I asked if the first God is the Random God and if da means no, would you give the same answer?”.

Remark: Notice that in the solution above we never figure out if da means yes or no. This is impossible given the current setting: If we were to figure out not only the gods positions, which contains $log_2(6) = 2.58$ bits of information, but also if da=yes for a total of $2.58+1 = 3.58$ bits of information, we would need more than three binary questions which only give us $3$ bits of information.

Online Mafia Rule book

2020-04-02T00:00:00+00:00

Game Overview
Mafia 2.0 (Ongoing)
2nd day votes
3rd day message
3rd day votes
4th day message
Mafia 1.0 (Finished Game)

Game Overview

There is a narrator that will guide the game. Everyone else will be randomly split into 2 teams: the Mafia and the Townspeople. The Mafia All know each other and want to pretend to be townspeople while killing of the townspeople one by one. The Townspeople want to discover who among them are part of the Mafia.

Requirements

WhatsApp Group Chat

Setup

The moderator randomly distributes the characters (listed below). There are 2 stages, day and night, which keep alternating until everyone from one of the side dies. The game starts in the day phase, in order to avoid having some character die before they have an opportunity to play.

Night: Each of the characters practice the role they were assigned. The essential role that can never be missed is the one performed by the mafia: They choose a townsperson to kill. During this phase the WhatsApp Group will have messages disabled.
Day: The narrator announces all the events that happened during the night. After that, messages are enabled and the townspeople start debating in the chat about who they think the mafia are or are not, based on everything that happened during nighttime. As people debate each selects the person they want to see lynched, and once someone gets a majority they are selected to be lynched. The narrator announces the character they were assigned, and we move to the night. When someone dies they are removed from the chat to keep it clear who is still playing, and to avoid comments from the departed :P

Characters

MAFIA: You are Mafia. Do not ask questions about your role in the group, only to me directly. Every night you and your mates will select someone to kill. That person dies unless the doctor chose to save that person.
GODFATHER: You are the godfather. Do not ask questions about your role in the group, only to me directly. Every night you and your mates will select someone to kill. That person dies unless the doctor chose to save that person. If you don’t decide together on who to kill, you have the final vote. If the cop investigates you during the night, he sees you as a normal victim.
RECRUITER: You are the recruiter, playing for the mafia. Do not ask questions about your role in the group, only to me directly. Once in the game you can propose another player to be recruited with a personalized text. That player can either chose to join you and the mafia, or reject the offer. They will not know who you are.
LURKER: You are the lurker, and play for the mafia. Do not ask questions about your role in the group, only to me directly. Every night you can lurk around someone’s home, and you will know everything that happened with them.
DETECTIVE: You are the detective. Do not ask questions about your role in the group, only to me directly. Every night you can choose a person to investigate, and i will tell you if that person is mafia or not. The godfather appears as a normal person.
DOCTOR: You are the doctor. Do not ask questions about your role in the group, only to me directly. Every night you choose someone to save. If that is the person the mafia chose to kill, that person does not die.
PROSTITUTE: You are the prostitute. Do not ask questions about your role in the group, only to me directly. Every night you chose someone to spend the night with. Any special powers that person might have used don’t take effect.
DRUNK: You are the town drunk. Do not ask questions about your role in the group, only to me directly. Every night you chose someone to drink with. That person knows they got drunk, and any special powers they had are not used. Neither of you remember the previous night.
BAKER: You are the best character anyone could be, the baker! Do not ask questions about your role in the group, only to me directly. Every night you chose a person and send them a cake, with any description your sick little head can think of. Game cues, plain insults, everything is welcomed :D I hope it wasn’t too obvious that I love this character. The cake you send will be seen by the entire town.
AGORAPHOBE: You are the agoraphobe. Do not ask questions about your role in the group, only to me directly. Due to your fear of the mafia, every day you can choose to hide in someone else’s home. If so, actions target at you will not take effect, but whathever happens to the person you are with will happen to you too.
TOWNSPEOPLE: You are a towns person. Do not ask questions about your role in the group, only to me directly. Even though you have no special powers at night, your role is to find the mafia: Interpret everything that happened during the day to figure out who is telling the truth and who is manipulating the game.
TRAITOR:

Practical notes

Couples in the group. Do not reveal your character to each other. That kind of ruins the game.
Votes must be written in this chat with two equal signes, such as : “== I Miguel vote to lynch “, and nowhere else. This is done in order to make vote counting easier (i.e. I can just search for “==” to count the votes). Messages without == do not count as votes.
People who haven’t voted yet count as having voted in themselves.
Votes cannot be removed, only changed to another person. (i.e. I can’t say I take may vote away from person X, but I can say that I vote now for person Y, in fact removing my previous vote).
I will be counting the votes. If there is a majority that person gets killed and the role is revealed.
The first person to vote is a personal hero of mine for kick starting the votes.
If after 2 days there is no majority, I will lock the possible lynchees to those who already have votes from others. After that every day I will be removing the least voted person until one is left.
Don’t worry about knowing all the game rules, just vote and see how it flows :P
If you want to ask questions about your role or anything that might reveal your character, ask the question to me personally. I will post the answer in the group chat for everyone to read.

Mafia 2.0 (Ongoing)

We are currently in the day phase. Since there are 20 players, we need 11 votes on the same person for that person to die.

Original Roles

1 Godfather (Mafia)
1 Lurker (Mafia)
1 Recruiter (Mafia)
2 Normal Mafia
1 Detective
1 Doctor
1 Drunk
2 Bakers
1 Agoraphobe
8 Townspeople

Players (15 alive)

Players with their role in parenthesis are dead.

Albert (agoraphobe)
Alexander (townsperson)
Ananth
Brigi (drunk)
Cheyenne
Claudia
Daria (townsperson)
David
João (townsperson)
Julian
Juul
Keith
Lennaert
Mar
Nadine
Natalia (mafia)
Pedro (townsperson)
Raluca (townsperson)
Ricardo
Sharon (townsperson)

1st day message

Welcome everyone to мафия town. For those wondering what it is like to live here, have a quick peek at https://www.youtube.com/watch?v=EhdneGEWU98 . We know that among us there are villains and there are heroes, but do we know who is who? We assemble in our main square to discuss it, in the hopes that together we will not perish!

1st day votes

(11) Daria: Daria, Nadine, Joao, Albert, Claudia, Lennaert, Ananth, Pedro, Julian, Cheyenne, Sharon
(4) João: Raluca, Ricardo, Sharon, Mar

2nd day message

Good morning good town of мафия. Apologies in advance for the unusually long night: this was due to a missing villager Brigi. Did she go missing because she trusts her mafia members to make decisions on her behalf? Is she bored of her role as a goodie do good townsperson? We don’t know. What we know is that we found ALBERT dead in the town square, his body choped up to pieces. It seems fitting of his role. Albert was the AGORAPHOBE. His fear of the mafia was justified. If only he had hidden this night in someone else’s place he would be here with us today. The town goes to the bakery for a warm coffee and cake to enjoy the first days of snow in the city and with it there is a gift. Curious, the town sees a label: “To Joao”. He does the honors of getting his rightful gift, a huge Dobos cake with a single phrase: “Life gives you a new opportunity”. Will the town have the possibility to play around and throw snow balls and build snowmans or will the mafia ruin another innocent life? While pondering this we find a second cake, directed to Ananth, stating “next party we will clean 😂”. Do we have two bakers in this town?!? With Albert dead and Brigi missing, we are as clueless as ever as to who the mafia is. Good luck!

2nd day votes

(6) Joao: Pedro, Claudia, Natalia, Nadine, Mar, Julian
(4) Pedro: Ananth, Lennaert, Cheyenne, Joao

3rd day message

A new day begins and the community is absolutely gutted, devastated. They lynched someone with conviction but it was a huge mistake. Not only that, Brigi was finally found after she left town, but she was found in tiny little pieces in our town square. She was the town DRUNK. Was her death the result of some inebriated misadventure with the mafia? We do not know. Town goes to the bakery to burn their sorrows with sugar. The baker is crying, feels guilty for this error, with a face full of tears she has a gift in her hands. The label says: “To Ananth”. Ananth opened the box and the town can see there is a cake with the shape of… Joao’s head! It was hard to recognize him and it’s full of blood, friends started to cry and there is a message: “Don’t forget me, we need justice”

3rd day votes

(8) Pedro: Mar, Keith, Lennaert, Ananth, Ricardo, Pedro, Cheyenne, Julian (2) Keith: Cheyenne, Claudia

4th day message

There is no time for condolences. This has been a silent and vacant city and it will be a mafia town soon if we do not do anything. Natalia has left us and Sharon has been killed.

People woke up today and they are heading to a protest in front of the detective’s office. All the community is coming out and villagers are very angry, they are asking for help. They look at each other trying to find answers.

There is a proposal for a new security law, which has to be approved by the mayority. All members have to write a public statement to explain which is their role and indicate 3 suspicious people with their reason by the end of the day to ensure mafia is removed and they live in a safe place. Will the town approve this law?

Suddenly people throughout the street were swivelling around to see an angry scarlet envelope that appears on Lennaert’s door, it is a Howler cake. The physical temperature of the Howler begins to rapidly increase upon delivery and it will explode if left unopened for too long. Is it true Lennaert is detective? Will the power of the Howler reveals a secret? He comes out and once opened the letter cake booms with the voice of the baker which is magnified beyond all belief. The message says: We’re gonna die. We need a proof that you are sincere, don’t lie, we dont trust anyone now. We don’t want to think you can be part of the mafia.

Mafia 1.0 (Finished Game)

The current phase is daytime. Since there are 15 players, we need 8 votes on the same person for that person to die.

Original Roles

3 Mafia
1 Godfather
1 Detective
1 Doctor
1 Prostitute
2 Bakers
6 Townspeople

Dead People

Istvan Meszaros (Baker)
Cheyenne (Townsperson)
Nadine (Townsperson)
Ananth (Mafia)
Fons (Mafia)
Pedro (Townsperson)
Agnes (Mafia)
David (Prostitute)
Tushita (Baker)

1st day message

Welcome everyone to the town of dies-ville. This is where people come to die. The sun rises and we are shocked to find something unusual. In the center of our little plaza there is a dead body. We look at it, disgusted, and it is the body of Zoltan! He left us but his death was not in vain, because with his death comes the awareness that there are criminals among us. We must ensure they do not survive. As such, in this ironically sunny day, we must select someone to die. Will it be you?

1st day final vote count

(8) Istvan Meszaros: Istvan, Antunes, Pedro, Fons, Nadine, Ananth, Claudia, Agnes
(2) Antunes: Cheyenne, Tushita
(2) Ananth: Luiza, Anastasia
(1) Agnes: David
(1) Fons: Andi
(1) Pedro: Mar
(1) Andi: Pedro

2nd day message

Good morning everyone! A beautiful sunset rises upon this blood shed town only for us to walk into our plaza and see… no one died! What a beautiful day it is indeed. The town goes to the bakery to enjoy it’s death free night, and with it there is a gift. Curious, the town sees a label: “To Pedro”. Pedro does the honors of getting his rightful gift, a lovely middle finger up shaped cake with a single phrase: “Stay at home or die”. Will Pedro give into this threat and follow his quarantine despite the advice of one of the worlds greats, Mr Dolan Trump, or will he rebel and jeopardize everything by venturing into the outdoors? For now, the time has come to look into lynching a mobster. Will we repeat the mistake from our previous day just to find out that we lynched one of our own?

2nd day final vote count

(8) Cheyenne: Tushita, Fons, Joao, Andi, Ananth, Luiza, Agnes, Claudia
(5) Ananth: Nadine, Cheyenne, Mar, David, Pedro

3rd day message

Good morning everyone! A sad day arises, not only because of the CoViD hovering over everyone’s heads, but also because the doctor has failed us this night in his endeavors. The honorable Nadine Helmi was found hanging by her neck at her home. An house search found nothing. No detective network to get information, no magical doctor pills to bring the dead back to life, nothing. Nadine was a TOWNSPERSON. At least we have a cake to console ourselves with. We all move to the bakery and we have a cake addressed to Ananth. It is heart shaped. Love is in the air. Ananth goes for it, butterflies in his stomach as he wonders who the mystery cake sender is. He can’t wait to get to know her (damn, does he hope it is not an him). He goes for the cake and his dreams shatter. Magic takes over the cake the second Ananth touches it, changing it’s color to the darkest black anyone has ever seen. From within the black, 3 words pop up in blood red: “YOU ARE WRONG”. Ananth cries.

3rd day final vote count

(8) Ananth: Pedro, Claudia, Luiza, Joao, Andi, Tushita, Anastasia, Ananth

4th day message

Good morning everyone! A new day dawns upon our faces, with the hope of a Mafia killed but the near certainty of another victim falling pray to the mafia. Our doctor did a wonderful job the previous night, can he repeat his prowess? We wake up in hope only to see cut toe in our town plaza. A body part is a seal of something dark, but a toe? Pedro shows up with another toe. João finds a finger. Anastasia a ear. The town goes around finding chopped up body parts like it’s easter. We assemble the puzzle pieces and start building the full picture. After a fun day out playing puzzles and doing picnic, wefind the puzzle solves to be Mar. She was dead, and not only did we solve the puzzle of her body pieces, we also solved the puzzle of her role. She was a TOWNSPERSON. For the picnic, we all had a cake, as usual. This time, it was addressed to João, with the shape of a dead horse’s head. Fitting

4th day final vote count

(6) Fons: Pedro, João, Claudia, Andrea, Tushita, David
(3) João: Fons, Agnes, Luiza

5th day message

Good morning beautiful people of dies-ville. Another day arises in this death torn town, having had a short night after so many deaths. Wait. It isn’t death torn today. We arrive in town and nobody is dead. Could it be true? Have the mafia taken an holiday? Have they hidden their body too well for us to find it? Turns out they just hidden it, in an attempt to be discreet Pedro’s body was hidden under the Plaza’s marble floor, his body crushed between the dirt below and the marble above. Pedro was a TOWNSPERSON. As such, we relocate to the bakery. I could describe the cake, but today the baker’s words surpass my writing skills. As such, I will provide you with a direct quotation:

It’s a beautiful cake with a smiley face which smells heavenly and makes the receiver share it with everyone in town. It makes everyone think of their favorite memories and will always remake/refill itself making it an endless/never ending cake. After finishing their first slice, people realize their mouth is dripping blood. Turns out that the cake has turned to blood and it hass written “liar liar pants on fire, next time you won’t get an orgasm”.

Have a good day everyone!

5th day final vote count

(5) Agnes: Claudia, Andi, João , Tushita, Anastasia

6th day message

Good morning people. Easter is not a time for misery and death: it is a time to be reborn and where we can cherish our closed ones, praise their well being and hope that no evil exists. Fate decided that was to be the case in dies-ville as well. No one died. No one suffered. All is good. We went to a bakery to celebrate.We get our cake. It’s a cake with a magnifying glass, classic Sherlock Holmes hat, his smoking pipe. there are a bunch of jumbled letters written all over it. The moment João touches the cake, it rearranges and says “ Your wish is fulfilled, but you suck as a cake detective”. What is the baker trying to say?

6th day final vote count

-(4) David: Claudia, Anastasia, Andi, João -(1) Anastasia: Luiza

7th day message

Good morning sweet people of dies-Ville. After a long exhausting night where everyone was doubting each other, wondering: Are you the godfather, the godfather did not stop in his relentless search to KILL OFF EVERYONE, So who did the godfather kill? Poor Tushita. We found her fallen forward on the street, single shot to her chest and her jaw dislocated. We can’t be sure how it happened, but I believe it was something like this . She didn’t leave us without baking her last cake though, as Tushita was the BAKER. It’s a chocolate cake for Andi which says “let’s end this now”. What did Tushita mean by this?

Marginal Losses

2019-06-25T00:00:00+00:00

While reading this article one paragraph struck me:

The idea is simple. Sometimes markets produce negative impacts that are not accounted for in market prices. The market participants who produce the impacts do not pay for them; society absorbs the costs. (Air pollution is a good example.) These “externalities,” costs external to markets, are considered a form of market failure.

Ĩn the case of environmental costs many solutions have arisen, the most famous of them being carbon taxes and the cap and trade system. However developing them was the result of results tailored for industries polluting the atmosphere. In a more general setting, can you improve markets such that these externalities are taken into account without the need for ad hoc fixes such as specially design taxes and other mechanisms?

Intelligence, Money, Fun

2019-02-03T00:00:00+00:00

Text serendipitously found in the web

I asked my dad if I have ever made him cry in front of me before, because I don’t remember ever seeing him cry. He said, “Once”. He told me that when I was 3 years old, he laid out a pen, a dollar, and a toy of some sort in front of me. He wanted to see which one I would pick. I think that a lot of Chinese people do that… It represents what you’ll value most when you grow up. Like the pen is intelligence, money, is well, money, and the toy is fun.

He was just doing it out of curiosity and boredom. It was interesting for him to see which one I’d pick anyway. He said that I just sat there and stared at the items. He sat across from me and waited patiently. According to him, I crawled towards them, he held his breath, and I pushed everything aside and went right into his arms. He didn’t realize that he was one of the choices. And that was the first, and the only time I made him cry.

Salem Witch Trials

2019-01-28T00:00:00+00:00

Found on the Web

One of my friends told me about a powerful lesson in her daughter’s high-school class this winter. They’re learning about the Salem Witch Trials, and their teacher told them they were going to play a game.

“I am going to come around and whisper to each of you whether you’re a witch or a normal person. Your goal is to build the largest group possible that does NOT have a witch in it. At the end, any group found to include a witch gets a failing grade.”

The teens dove into grilling each other. One fairly large group formed, but most of the students broke into small, exclusive groups, turning away anyone they thought gave off even a hint of guilt.

“Okay”, the teacher said. “You’ve got your groups. Time to find out which ones fail. All witches, please raise your hands.”

No one raised a hand.

The kids were confused and told him he’d messed up the game.

“Did I? Was anyone in Salem an actual witch? Or did everyone just believe what they had been told?”

And that is how you teach kids how easy it is to divide a community.

Keep being welcoming, beautiful people. Shunning, scapegoating and dividing destroy far more than they protect. We are all in this together.

Magnitudes

2018-08-30T00:00:00+00:00

Someone might have a million GBP in the bank. There are over 7 billion people in the world. The US current debt is over a trillion USD dollars. What do all these numbers have in common? They sound like a lot. But when reading about them, that is the only similarity. Even though a company being worth a million or a billion dollars makes a huge difference, it is very hard for someone to perceive the difference on first notice since the idea of a million, a billion and a trillion are not used in our daily life.

So what are you suggesting?

There is a way around this problem. We have well-developed intuitions about smaller numbers since they appear frequently in our life. Hence, we can convert the giant numbers above to smaller numbers by thinking of magnitudes, or even simpler, how many digits the number has.

And what do I do with that?

Now that we have numbers written in magnitudes, we can easily compare them. For example, if our unit is euros, we can consider which items are in each magnitude:

Cost of a meal, public transportation ticket
Clothing
Furniture, rent, plane tickets
Family holiday
Car
House, wealth of smaller companies
Launch of a reusable space rocket
Airplane
World population
Ethereum’s Market Cap
Cost of the British National Health Service
Value of assets managed by the world’s largest hedge fund.
National debt of the United States.

Every time we see a new number, we can think about where it lies in this magnitude list and compare it to what we already know. That way when we see that a million dollars were spent on a new car fleet for government officials, hopefully we’ll realize that a million is about the correct value for such a fleet, instead of being outraged at the sound of a million dollars being spent by the government in something apparently unnecessary.

Can you build a detailed magnitude list for yourself?

Edit: This Ted-Ed Video shows this concept in action. Enjoy!

上海

2018-08-29T00:00:00+00:00

Shanghai. In truth I am currently in Beijing’s Capital International Airport, sitting down at a Starbucks, pondering about the last two months, and waiting for the second leg of my flight home, which will leave me in Amsterdam before I’ll be able to on-board the last flight to Porto.

A few months back an email popped up in my inbox. It advertised a summer in China spent teaching. I have only tutored before and had no experience of teaching. They covered the transport costs and paid a modest amount for the experience, so it looked promising nonetheless. Without thinking much about the topic, I sent my CV. Soon I had my first phone call in order to explain why I was interested in this opportunity, and what were my thoughts about teaching. Since I have heard many things about the educational system in Asia, I wanted to see it first hand, and having teaching experience would be a quick way of quickly grasping the differences between the Western and Oriental educational styles. So I moved on to the second interview (with the naked mathematician) in which I was challenged to present a topic of my choosing in 10 minutes. It was fun preparing a lesson in detail, and it was a good way of them assessing my ability to teach. In the end, the lesson went smoothly and I got my offer to join this program. After some struggling with the Portuguese Embassy in London I finally got my passport and Visa in order to fulfill my summer plans.

I think we know all we want to from your application process, but what about the internship itself?

The internship would take place between the 2nd July and the 21st August. I booked my flights so that I would arrive three days earlier, in order to explore Shanghai before work starts. During my flight I saw for the first time the Eiffel Tower. It was a shame that I saw it from the window of a plane, but I am confident I will see it in person sometime soon. The long flight continued all the way to Shanghai and for the first time I would see China. Without a working phone and without understanding a word of Chinese, my first goal was to find my way to the school Campus. I had the address, I got myself into a taxi and just gave the driver the written address. We were driving for over an hour (which I didn’t expect), and the driver tried to make conversation on occasion. It was hilarious given that he didn’t speak English neither did I speak Chinese. At one point we managed to discuss how I was going to pay, to which I showed him my debit card, which I discovered isn’t that useful to pay for taxis, so after he complained for a while we stopped at an ATM, I got the cash, and we continued the trip. I made it alive to Campus, and met part of the staff, which received me very well.

That seems like a good start to the summer

It was, and upon arriving I was surprised at how huge and well protected the campus was, having security guards, barbed wire and cameras everywhere. At first it felt like I should be worried about being robbed in the streets around us, but soon I found out my concerns were not justified. Even though I arrived 3 days earlier in order to explore Shanghai, my lack of discipline regarding JetLag resulted in me sleeping most of the day and being wide awake during night. During those 3 days staff was bringing me breakfast to my room, and on Sunday I was awake but sleepy 15 minutes before Catherine brought me breakfast. Since I knew I was almost falling asleep, I left the door wide open so that she could just leave my breakfast in and I would eat it as soon as I woke up. I ended up not waking up until 6 PM, by which point breakfast was no longer something I could eat. Thankfully the next day the remaining interns (as we called ourselves) arrived and my sleep scheduled adapted to what I had to do.

So who are these interns?

The full group was me, Adam, Andrew, George, Issac (not to be confused with Isaac), Nayan, Maria and Jian. The eight of us would be responsible for preparing lessons and taking care of all the teaching for the following 3 weeks. This would prove to be a fun challenge. I ended up spending most of my time with George, who was both my room mate and my maths mate. I don’t think I could have had a better coworker: both hardworking and undoubtedly prepared, we made every class work smoothly and efficiently. It was fun working with him and the others. We had a small working place in the office, which was in fact a classroom with the tables rearranged to look like an office. All the interns and remaining staff worked together there and we would end up spending many late nights preparing materials together, listening to music, playing games and just having chit chat until it was time to go to bed.

That’s a very lame introduction of the interns.

It is, so here’s more.

Adam played the guitar and sang surprisingly well, almost as well as me, but the thing that always surprised me was his pop culture references. Always spot on.

Andrew had the girl fan-base that Issac wish he had. I could introduce both of them better, but I prefer to leave the reader with just that idea.

Nayan was an half native since he studied in Shanghai during high school. He introduced me to many of the local flavors I wanted to taste, and always had the drinking games prepared to have everyone say more than they should. The games worked well. Mostly.

Maria was always keen on exploring Shanghai and making the most of this summer, wanting to always make the best of each situation.

Jian was cool and calm. He had the vibe of someone who has everything he needs to be happy sorted, and didn’t need others to approve of his condition. I aspire to share in that way of being as much as he transpired to have it.

That seems decent, but I haven’t heard about any teaching yet. Did you actually do any work during the summer?

We did, so let’s start in order. During the first week we met our inductee, Dara, who was responsible for introducing us to the program and making sure we had all the information we needed to do our work properly. It was a crash course both on getting us all to know each other by asking silly questions such as “Would you rather be hairy all over or bald?”, and actual work, but it turned out to be just what we needed to get started, and her energy transpired to us, making it all work. Little did we know how much we would miss having someone responsible for keeping the interns up to date, a need which hopefully will be satisfied in following years.

Good to know you had a good inductee, but that is not your work. What did you do?

My responsibilities consisted of preparing teaching materials, teaching, correcting / rating personal statements and occasionally do presentations to the students. So let us go through them.

Teaching: We were given some materials used in previous years, but other than the topics and some rough guidelines we ended up preparing a lot of the materials ourselves. This worked well because we ended up knowing exactly how to teach the material. It turns out that to teach a topic, it does not suffice to know it well, you need to imagine what is like not knowing it, and how to best introduce it to someone who has no idea what you are talking about. It is a great exercise in understanding the fundamentals of a topic, and I recommend teaching as an exercise to everyone. There are other parts which are more classroom related, but after this summer I stand with a belief I had before, which is that the greatest gift a teacher can give a student is curiosity. I can go through hours of boring practice, after which a student will be proficient at a topic, more so than experts in the area, and yet leave with a feeling of disgust for the topic, never wanting to see it again. However if you successfully fire up their curiosity, they’ll be engaged for a long time trying to understand how it works, getting a better understanding than an ordinary student would get. I was satisfied to have one of my students write to me, at the end of camp, that “I love how your eyes twinkle when you meet an interesting problem”. Sharing that mathematics, or any subject, is still interesting after four years spent not doing much else is what I wanted to convey to my students: any topic can be interesting if you approach it in the right way, and I am glad I could contribute to that part.
Personal Statements: This was the boring part. I feel, maybe wrongly, that in mathematics personal statements aren’t very important since there is so much that interviewers can test you after twelve years of learning. Nonetheless, it was part of the application process for University and part of my responsibilities to help my students with the entirety of their application to University, so I helped. Even though this was the least enjoyable part of the work, it was more bearable to see all the cases of Chinglish and the hilarity that ensues. I will not share specific examples since these references concern privately written personal statements, but consider that after two months of me trying to learn Chinese, I still can’t pronounce the difference between 吗 (question marker), 马(horse) and 妈 (mother). I can only imagine the funny sentences that I might have said to my students. If any of my students is reading this and thinking that they made such mistakes, I just want you to believe one thing: life is full of mistakes and mistakes is something to be proud of. That is how you learn and grow, and as long as you work hard not to make the same mistake twice, you are good to go. As a side note, it took me six months upon the start of University to understand the difference between saying “hill” and “ill” since in Portugal we don’t have the “h” sound. After many weird looks and giggles from other people, eventually I improved my English and now I understand and control the two sounds. But this improvement only happens if you don’t shy away from making mistakes. So go ahead, make a fool of yourself, and know that even if people laugh at such silly mistakes, they are harmless and you are learning a lot in the process.

You forgot to talk about the presentations.

They are not that interesting.

Fair, so what else did you do?

We explored Shanghai a lot! We went to the Bund, the Pearl, had hotpot, KTV, and traveled outside Shanghai during weekends too. But first let me explain KTV since I’ve never seen it before. KTV is basically karaoke with no one but a group of friends in a small room. This sounds simple enough, maybe similar to sing star in a friend’s house, but it is so much more than that in Shanghai, with entire floors dedicated to KTV rooms, with a wide selection of songs and an environment that is hard to recreate in a friend’s house. We ended up doing it several times, including for mine and Issac’s birthday, and in the last day with my first batch of students.

What about outside Shanghai?

We visited Suzhou, Hangzhou, Beijing and the Shaolin Temple. Those were some four amazing visits. We saw temples, climbed mountains, walked the Great Wall of China, explored the forbidden city, revisited history in Tiananmen Square, scared old ladies, immersed ourselves in nature and in cities. So much was done and felt in such a short period of time, it is hard to describe all that we did.

Wait, wait, wait. Scared old ladies?

Yes. This is a funny albeit short story. On our trip to Beijing we rented an airbnb for four nights. The apartment was awesome, I was particularly fond of the stains in the bed cover, the shower that wasn’t physically divided from the the rest of the bathroom and it would flood the entire bathroom every time one of us showered, let alone all six of us :P Anyhow, before all of this, we went to building 401 in a certain street in Beijing, and our room was 1605 (16th floor, 5th room). The corridor looked interesting, feeling like we were playing slender man in real life and our goal was to find home. We arrived at our room at midnight. Since it was so late our host just said that the key was in the door, and for us to walk in. We tried opening the door, to no result. We knocked, but no one answered. We tried to find the key under the carpet, but it wasn’t there. We found a small bell on the side and eventually a lady came to the rescue. It was late at night so, understandably, she didn’t open the door. The conversation went something along these lines:

Lady:Hey.
Us: Hey.
Lady:Who are you?
Us: We booked an airbnb for tonight and this is the airbnb we booked. Can you open the door?
Lady:No you are not staying here for the night.
Us: We booked it online and paid for it.
Lady:This is my home.
Us: But we booked it online, we have the booking receipt here.
Lady:No, this is my house, I’m not letting anyone inside.

By this point we asked Jian to check the booking. Turns out building 401 was the owner’s building, we were sleeping in 406. Our bad. Without her ever opening the door, we apologized, and went into the correct building where indeed the key was on the inside of the door, the door not being locked. For a while we wondered what went through the ladies mind. 7 foreigners, young and able, knocking on her door at half past midnight demanding to get in cause that was our apartment for the next four nights. I hope she managed to sleep that day, but i am not sure she did.

Poor lady.

Looking at the bright side we might have spiced up her life. That’s always good :D

Not by scaring people.

Fair. Anyhow, we explored China a lot and we learnt a lot in the process.

Such as?

I’ve mostly been talking about what I did as a tourist, but we also got to experience what it is like to live in China. The first thing that many might notice is the lack of toilets as we know them in Europe. Instead, we have squat toilets, which consist of a hole on the floor. A fancy whole, built exclusively for the purpose, but a whole on the floor nonetheless. Surprisingly enough, I honestly think I am going to miss squat toilets. They are equally useful, and nowhere nearly as much of an issue as I thought at first. And my legs got stronger too. And better for bowel movements. They are better all around. Maybe I should get myself one back home :D

Secondly, WeChat and Alipay. They are everywhere. All of your finances managed on your phone, everything centralized. This is a sensitive point if you consider how much control the government has about all your spendings, but from the point of view of a tourist who used the app for two months, it is so practical. No money to handle, just a bar code and everything follows swiftly. All payments to friends, shops, wages, everything. It is so much more convenient. And then there are the apps inside that make it all even more useful, most of which sadly I didn’t use. Renting bikes, where 1 euro allowed you to ride for 7 hours, or movie tickets, or ordering food, or anything. It was like looking into the future. I wonder what it would take to bring the tech back home, because I definitely enjoyed having it at my fingertips.

Thirdly, and probably the most obviously, the language barrier. I’ve been to Mozambique, Panama, Brazil, and a few European countries. What they all had in common was that the population was generally fluent in either Portuguese, Spanish or English. Any of these are enough for me to get around easily, but in China I had none of them. You might find the occasional English speaker, and definitely there were many of them in the program, but that was not the general observation outside the confinements of campus. It was a new experience to me struggling to convey simple messages, and having to signal for most things I wanted. I’d like to say my Chinese helped, but other than very basic things I failed at communicating. I need to practice more.

As an interesting side story, today on Beijing’s airport I started talking to one of the security guards at one of the checkpoints. She kept asking for something I didn’t understand. I gave her my passport, which she refused. My boarding ticket, which she also refused. At a certain point she just gave up and told me to continue through the checkpoint. I believe I got through a checkpoint without the proper checks just because she thought “I’m tired of trying to speak, I’ll just let him go”. I still have no idea what she was asking for.

Is this all?

Definitely not. I mentioned a 3 week camp which I taught, but after that one we moved to a new one, where we were joined by more interns: Adina, Cam, Henry, Leo, and Richard smells. Henry ended up working a lot with me and George, and was a great addition to the team. And the students liked him a lot, which me and George always wondered about. Obviously we weren’t jealous of it. Obviously. He saved me from a lot of work in the last day. I needed him to take over a class last minute, even after me and George confirmed that he wouldn’t have to, and he did so smoothly. What a guy. That together with his keenness on being a good teacher made him a pleasure to work with. I remember one day him teaching modular arithmetics. Since he studied computer science, he wasn’t as fluent in some parts as much as a mathematician, so I explained those and the attention and effort he showed then is something that a teacher wants to see in every student, and it makes teaching worthwhile. Seeing him doing all that so that he could do his job to the best of his abilities was a delight.

We get it, you liked working with George and Henry, the mathematicians, but that’s boring. Say interesting stuff.

Fair. I liked the bunch.

Leo was the party animal, which was harder for me to relate since I’m not as fond of clubbing, but his random trivia was consistently interesting, and always left me with the question “What possibly reason could there be for him to know such random things?”.

I particularly enjoyed Adina because the two of us remained strong when climbing the mountain near the Shaolin Temple. For hours we walked and walked, making it to the top and enjoying the views that, in the end, only five of the interns went up to see. she was good company during that long walk.

Cam’s smile was interesting since all the students agreed that it was a fake smile. I did not see it as fake, and he himself claims that was not a thing ever before, but his students disagreed. I can only wonder what sort of smiles he had during class that transpired such fakeness. I really wish I had gone to his classes.

Richard. Smells.

So far you only told us about the interns. Did you work with noone else?

We did. There was a second inductee, Tom, but I ended up spending little time with him since his responsibility was the new interns. We had Yao Yao and Potter who did an amazing job of coordinating camp with so little to go on. Kudos to them. We had Flora who made sure all the interns were well taken care of. And all the staff who worked wonders to make sure everything went well, be it Alex with IT, Betty with finances, Catherine with accommodation and weekend trips, and everyone else who worked in the background. But there’s a group of people that us interns are specially grateful to. The Teaching Assistants, or, as we called them, the TAs. They were there to reduce the teacher’s workload, by supervising students in the morning and evenings, by marking the students work, and doing anything needed in the moment (Thank you Cindy, Helen and HD! ).

So they helped you with the workload?

Yes, but also with all the breaks and jokes and stories that made the entire program enjoyable. Tia and her bear, or should I say beer, which I took very good care of, but in the end got torn apart and lost somewhere dark. Alice and her curiosity about Measure Theory. I feel like a nerd describing the “good times” as me teaching mathematics on top of the teaching I already had to do. But then again, maths is fun, so there’s that. There’s all the songs we sang in KTV. Tia’s brother was a delight. I miss you, 小胖.

Cool, glad to know you enjoyed something else other than work

There’s another thing! I do enjoy my maths, but I enjoy sports a lot. I might not enjoy aimless exercise, and by aimless I mean exercising in solitude, but be it football, handball, basketball, or any clash of physical prowess gets my interest, and as such I really enjoyed our late evening basketball and football matches. But most of all, I ended wrestling as well. I haven’t done that in ages, but by chance me and Issac ended up wrestling to see who got the middle bed in the Shaolin temple. I need to wrestle more often. Maybe I should try boxing. Also, winning felt good :D

You’re just saying this to tell us that you won when wrestling?

Obviously winning is great, but I genuinely enjoy exhausting myself physically. Do you not agree?

I do, I do. What else did you do?

I watched the students in the Shaolin temple learn martial arts, and do double back flips as if it was the most natural thing in the world. I was amazed by the theater session they showed us where their stage was the entire mountain. Seriously, there were people and lights all over the mountain, from top to bottom. I still don’t understand how they created the illusion that a building, which was just in front of my eyes, rose from the floor. No way they actually raised an entire building, but it was there, and it was going up. I went to Shanghai’s concert hall, which was gorgeous, even though the performance was a bit disappointing: I feel like the pianist butchered some great classics. I spent an afternoon reading in Pudong’s library, which is bigger than any I’ve seen before ( Thank you Flora for taking me to so many different places and being great company for my last two days in Shanghai ). I had dinner with the rest of CamExpress at the rotating restaurant in the Pearl, which oversees the never ending city. I watched national productions, namely “Dying to Survive”, which was a lot more emotional than the movies I’m used to watch. The weight of that movie was balanced by the lightheartedness of the comedy Mr Billionaire, which made me laugh all the way.

That is a long list.

It is. And there is much more. They were a packed couple months. I experienced a lot. Both work wise and personally. I made friendships that will last. There was conflict. But life is about taking the good and the bad.

Are you trying to get a moral out of this?

I wasn’t planning on, but I kind of feel like I must now. Because there is always bad and good in any situation, and we only appreciate the good after feeling the bad. We assess our well being by comparison. So there can be enjoyment in bad situations. There’s a choice between getting angry and frustrated at whatever bad comes our way, or making the most of it, and taking it as a learning experience. And whenever that bad happens, instead of spending your energy on frustration, it can be used constructively by working on it. By thinking on what can be done to improve the situation. By thinking on what can be done to make sure that it never happens again. By making sure you helped to solve another one of the many problems in this world. Because, in the end, “life is misery with a few dots of happiness”.

Echo Chambers

2018-02-23T00:00:00+00:00

Social Networks are meant to provide users with social updates, be it from friends or from the media. Given the information overload we can’t afford to read everything, and as such the platforms (Facebook, Twitter and many others) choose what we should read on our behalf. They employ algorithms that try to find the articles we find will most likely enjoy. An honorable attempt to achieve the not so honorable goal of keeping us using their platform for as long as possible.

So how should we change it?

The first changes I would demand are:

The users to be aware of how the algorithms used work.
Have a selection of algorithms for the user to choose from.

Change 1 can prove itself hard from a market competition point of view: the platform with the best algorithms should have more success, but revealing them would nullify any investment on these areas. This shouldn’t prove problematic since the best algorithms are already out-there up to tuning parameters. What I mean is that companies can say that they use a certain algorithm without specifying the parameters that made it work best. A great example of this in practice is the public release of Reddit’s ranking algorithms . I admire their initiative. Also, they cite this page which I found interesting for anyone doing rankings.

Change 2 is easy to implement but so far it seems uncommon practice. I’m talking at least Facebook, Twitter, Instagram, Pinterest and Snapchat (following this). For UX purposes it makes total sense to have a default algorithm so that users don’t have to worry about how each algorithm works and, to be honest, it’s understandable that most users don’t care. However the option should be there (somewhere in settings for users that don’t). There are many reasons for these, not least of them the fact that I want to have my feed better chosen than to just see the posts that I will more likely like.

Personally, my problem is echo chambers.

What are echo chambers?

In common usage, an echo chamber is a term to any setting which feeds you back with the things that you already believe. Do you like kittens? Here’s a 2 minute video of kittens purring. Are you Pro-Life? Then we’ll send you some more Pro-Life articles. Do you want stricter immigration rules? Then we’ll send you links to people who’ve moved society towards that goal.

We move from an innocent yet naive approach to social feeds to an echo chamber that feeds us with what we already believe in and prevents useful discussions. We need to discuss hard topics with the opposition. Yes, we need to be careful about politics, religion and soccer (never forget soccer), but we DO NEED to have this discussion. With tolerance and an open mind, but anything is better than avoiding this conversations and living in our own little bubbles, as if nothing else existed.

The Wall Street Journal published an interesting take on echo chambers by making a comparison of Liberal and Conservative news feeds. The results should cause outrage. People on both sides will leave their feeds with a feeling that everyone agrees with them, with no discussion and, more importantly, no agreement being achieved.

What do you propose?

By the tone of the post so far it should come as no surprise that we need better algorithms. Being shown posts with divergent opinions from the user will definitely increase churn rate, so platforms have no incentive to do it. My hopes lie in people becoming aware of the Echo Chamber problem and demanding a solution. Personally, a simple solution can be achieved: let me choose a better algorithm for sorting my feed.

Which algorithm do you want to see implemented?

I call this the Explorer Algorithm. As the name hopefully implies, it is meant to explore posts that I wouldn’t normally see. Let’s imagine back to the Liberal/Conservative divide. We can divide the users into 2 clusters. Now instead of choosing the cluster that I am in, I am shown the posts ranked as if they were shown to the opposite cluster. That way I’m seeing more useful posts.

This has an obvious problem. If I enjoy politics, the approach above might show me all about video games which is not what I want. I want to see divergent views on political matters. Hence we need to do some fine tuning.

Iteration 2: We cluster people by what they actively like/dislike, regardless of which of the 2 they choose. For example I would frequently like political posts supporting my views and dislike political posts against my views. Hence I would be clustered with people who are focused in politics. Having people clustered by topics of interest, we can redo the above. Considering only people within a topic, we cluster them according to likes/dislikes (now distinguishing the 2 events). After that I am shown the better ranked posts for the set of people outside my cluster. This way I see relevant ideas that go against my views, and with some luck I’ll learn and adapt my views for the better.

Cool, we just need to implement it.

Yes. That can be problematic though. Even though these ideas are socially good, they might increase churn rate as mentioned above, hence reducing the platform’s profits. It would also take some power away from the platforms to control what is shown. It is not an easy fight, but public pressure is a valuable tool at our disposal and it can be effective if we are consistent.

So what should I do?

Share your thoughts on those same platforms.
If you know someone working there (or even if you don’t ) get in touch with them sharing your concerns. Only their employees can implement these changes.
If you find a platform that follows these ideas start using them more, leaving a post explaining why in order to raise awareness of the issue. Do follow through with changing platforms since nothing convinces social platforms to change more than hurting their profits from advertising. So far only Reddit comes to mind having implemented the ideas above, but do share other platforms who also do so if you are aware of them.

The Public Goods Game

2018-01-29T00:00:00+00:00

Adapted excerpt from Dan Ariely’s Predictably Irrational

This thought experiment offers an interesting example of the phenomenon of the tragedy of the commons. Imagine that I offer you and three other people £10 each, which is yours to keep. But I also give you an opportunity to make more money. You can put as much of your £10 as you want in the “group pot”. Once all the players have privately decided how much of their money to put in the group pot, all the money doubles and then is split evenly among the four participants, regardless of how much money each individual contributed. How much of your £10 would you put into the group pot? If all four of you put in £10, the group pot would double from §40 to £80, be divided four ways, and each of you would take away a nice £20.

So lets So let’s say you put in your £10, thinking that the other players will do the same. Once the pot is divided, however, you only get £15 back, not @20. What happened? It turns out that one of the other players, Bernie, decided to cheat. At the beginning of the game, Bernie realized that he would make the most money if he withheld his £10 while everyone else contributed to the central pot. So Bernie kept his £10 while the other three players placed their £10 in the central pot, making the total £30. This doubled for a total of £60, subsequently split four ways (remember, Bernie gets the proceeds of the group pot, whether or not he contributes), and each of the three contributors get £¹5. Bernie, meanwhile, makes out with £25 ( his original £10 plus £15 from the split), more than any other player.

Now, let’s say you get a chance to play the same game again with the same players ( I give you all new £10 bills). How would you play this time? You don’t want to be too trusting because you know Bernie might defect again. So you only put in 4£. It turns out that the other three players feel the same way and make the same decision, making the total in the group’s pot £12 (Bernie again does not put in any money). The pot doubles to £24, which is split evenly among four people. Each of the three contributing players gets £6 (In addition to the 6£ they kept for themselves), while Bernie winds up with £16 in total.

Now, your trust has been eroded. You play a few more games, but you don’t put in any of your money.Each time you end up with the 10£ you started with. You don’t lose anything, but since you don’t trust others to behave in a cooperative way (and neither do the other players), you don’t gain anything either. In contrast, if you and the others had acted cooperatively, each of you would have wound up with as much as @20 per game.

Viewed this way, the Public Goods Game illustrates how we, as a society, share the public good of trust. When we all cooperate, trust is high and the total value to society is maximal. But distrust is infectious. When we see people defect by lying in their ads, proposing scams, etc., we start acting similarly: trust deteriorates, and everybody loses, including the individuals who initially gained from their selfish acts.

From Surviving to Living

2018-01-04T00:00:00+00:00

Suppose a company has 2 employees Anne and John. The company values Anne’s work 2 times more than John’s work. As such the company is paying Anne £1400 and John £700. This can be seen as fair, one person having double the money to spend and be happy since they are contributing double of what others are contributing.

However there is one problem: Both of them have survival costs, namely rent, food, eletricity and similar expenses. Assume that both Anne and John need £600 to survive. Subtracting survival costs, we get that Anne’s spending money is £800, but John’s is £100, like this:

	Anne	John	Ratio
Salary	£1400	£700	2:1
Survival Money	£600	£600	1:1
Spending Money	£800	£100	8:1

In practice Anne has 8 times the spending money John has. Those are 8 times more money spent on actually improving your life quality, having dinners with friends, buying books to read, doing hobbies they love.

The core idea, shared on medium by Conrad Shaw, is that each person has survival money and spending money. Survival money is needed to remain alive, spending money is used to be happy.

What are you claiming?

My claim is that a person’s quality of life is not correctly dependent on their productivity. A small improvement in productivity leads to large quality of life improvements. In other words, if your productivity is slightly below average, then your quality of life is a lot below average.

Can we fix the situation?

Given the premises, we need to make sure Anne’s spending money is double John’s spending money. We have a total salary of £2100. Both Anne and John need £600 to survive, leaving us with £900 to distribute. Hence John should receive £300 (half) and Anne £600 (double).

Then we would get the distribution:

	Anne	John	Ratio
Salary	£1200	£900	4:3
Survival Money	£600	£600	1:1
Spending Money	£600	£300	2:1

Now their Spending Money and their value to the company are in the right proportion.

So companies should take that into account when deciding salaries?

Unfortunately that would not work well. As mentioned in the beginning, Anne’s work is twice as important as John’s work. Hence for the company we would observe something similar to:

	Anne	John
Company worth	£1400	£700
'Fair' amount	£1200	£900
Company Profits	£200	-£200

The company would be paying Anne below her worth but not John, so they win from hiring Anne but not John.

Do you have an alternative suggestion?

Yes, taxes. After the company pays employees their worth to the company, then we have to pay taxes by filling IRS forms and all our obligations. Applying the above reasoning to all citizens, then we should get the correct spending money ratio’s.

How would we figure out how much each person has to pay at a larger scale?

I am going to skip the calculations, but doing them is equivalent to each person giving a percentage of their salaries to pay for everyone’s survival costs.

How would that work in practice?

I will give you an example. Suppose that we have a population of 560 people with the monthly income distribution below.

Salary	Number of people
£0	30
£500	100
£750	250
£1500	100
£2500	50
£5000	22
£10000	10

If we assume that £400 is the needed survival money then we will need £224.000 to pay everyone, from a total of £3.712.500 available. Hence the survival money is roughly 6% of the total income seen. Doing the calculations above we get:

Initial Salary	Spending Money	Adapted Salary
£0	£0	£400
£500	£470	£870
£750	£705	£1105
£1500	£1409	£1809
£2500	£2349	£2749
£5000	£4698	£5098
£10000	£9397	£9797

The Spending Money column is just a rescaling (by 94%) of the Initial Salary column. As such each person’s initial salary and spending money can be seen as fair. Lastly (for those of you with higher salaries), there’s an important column missing: The percentual change in salary:

Initial Salary	Adapted Salary	Percentual Change
£0	£400	Infinity
£500	£870	74%
£750	£1105	47.3%
£1500	£1809	20.6%
£2500	£2749	9.96%
£5000	£5098	1.96%
£10000	£9797	-2.02%

Hence by 10 people giving 2% of their salaries we are able to guarantee enough survival money for 560 people, while keeping the spending money approppriately sized.

So we should just tax everyone’s income to provide for everyone’s survival?

Yes and no. There are a few things that should be done before moving forward:

1. Trials. Many Basic Income Pilots are already being done to better evaluate the impacts of having an UBI, from the citizen’s willingness to work and what work gets done, among other things.

2. Consider funding alternatives. I have provided a single UBI funding idea, which I personally like but I am absolutely sure there are better ones that I have not considered. Given the variety of funding sources for governments this should not be problematic, it just needs to be considered.

3. Defining survival money. This amount varies between geographical locations and means available (if I have my own house I usually have to pay a smaller rent than if I had a landlord trying to profit). Finding the ideal amount for a country is not trivial.

Not as relevant but also important, I need to do the calculations above to actual income data and not just fictional values like the ones I created. I am curious to see what the impact would be in that case. If you read this before I have done that feel free to email me so that I do it.

Long Term Teaching

2017-12-29T00:00:00+00:00

This post is a reflection on a previous post based on our Cognitive Breaking point. I want to compare 2 examples on teaching. One can be called more academic, and the other less so.

So what are we learning?

We are learning how to compute the maxima and minima of functions. And I will try to teach it in 2 different ways. My questions to you, the reader, are:

Which one would you rather learn?
Which one would you rather see taught in schools?

Alternative 1

This is how you compute the maxima and minima of a function $f:[a,b] \mapsto \mathbb{R}$, with $a,b \in \mathbb{R}$ :

You calculate its derivative $g = f'$
You get the zeros of $g = f'$
We compare the values of $a$, $b$, and the zeros of $g$
To figure out which of those points are local maxima or minima we need to calculate $h = g' = f''$. If, for $x_0 \in [a,b], h(x_0)<0$ then $x_0$ is a local maxima. If $h(x_0)>0$ then $x_0$ is a local minima. If $h(x_0) = 0$ then we call $x_o$ a saddle point (it is neither a maxima nor a minima).

To do all of the above we need to be able to compute derivatives. When calculating derivatives the following rules apply:

\[(cf)' = cf'\]
\[c' = 0\]
\[(f \pm g)' = f' \pm g'\]
\[(fg)' = f'g + fg'\]
\[(1/f)' = -\frac{f'}{f^2}\]
\[(x^n)' = nx^{n-1}\]
\[sin(x)' = cos(x)\]
\[cos(x)' = -sin(x)\]
\[f(g(x))' = f'(g(x))g'(x)\]

If you remember all of the rules above you will be able to present any question posed in the class and exam.

Alternative 2

We are trying to find the local minima and maxima of a function $f:[a,b] \mapsto \mathbb{R}$, with $a,b \in \mathbb{R}$. One useful tool to do so is using derivatives. A derivative can be understood in the following way:

Picture the plot of the function $f$
Choose a point $x$
Take the line tangent to $f$ at point $x$
That line has a certain gradient $m(x)$
This gradient is the derivative $f'(x)$

In another words, the derivative is the gradient of change or speed of change at point x. As such we can relate this derivate to the notion of speed we built in Physics class. Now what is the second derivative? It would be the gradient of change of speed. In other words we can think of it as acceleration. It turns out that derivatives are fairly easy to calculate for most functions, but we’ll do so later on. Now, given this tool we have, we want to find out how to find the maxima and minima of $f$. If you’re picturing a maxima of $f$ in front of you, then we can see that the tangent line must be parallel to the $x$ axis. Another way to think about it is, like before, think of $x$ as time and $f(x)$ as your position. Being in a maxima or minima equates to being the most forward or the most backwards. SInce to get there we needed positive speed and to come back we need negative speed, than at the point when $f(x)$ is maximal we need zero speed, so we need $f'(x) = 0$.

Hence the local maxima / minima is attained when $f'(x) = 0$. However there is one exception. The idea of me moving forwards then backwards from the maxima only works if I can indeed move before and after the time $x$. In other words, if I’m at one of the endpoints $\{ a,b \}$ the above does not apply. Hence we can have local minima/maxima when $f'(x) = 0$ or at $\{ a,b \}$.

We still need to figure out what are the maxima and what are the minima. Thinking in terms of gradients of slopes becomes hard at this point, but thinking in terms of acceleration makes things understandable. Say you are at a local maxima. Then your speed must be going from positive (moving forward) to negative (moving backwards), so your speed is decreasing, so you have negative acceleration. Hence $f''(x) < 0$, since we saw that $f''$ represents our acceleration. Similarly, at a local minima, $f''(x) > 0$. If neither is happening then we are neither at a local maxima nor minima, so we will call it a saddle point. Imagine a saddle point to be a the point where your speed is zero (just like in maxima or minima), but you are not accelerating away from your extrema. You might have gotten to an extreme and stayed there, or you might be moving forward, stop (speed is zero) and then start moving forward again, so it definitely isn’t a maxima / minima.

Hence we figured out how to calculate the maxima and minima of a function $f$ .

What’s your point about the alternatives then?

I will try to answer two questions about the above alternatives. First, which one would lead to better grades? Secondly, which one would be better for students development, however vague that might be?

For ease of writing lets assume student A was exposed to alternative 1 and student B to alternative 2.

Regarding the topic above, the grading method is fairly straightforward: We give functions to the students, and ask for them to tell us the local maxima and minima. Given that evaluation method, student A is most likely to provide better results. By consistent practice we trained A to give fast and correct answers. Student B, on the contrary, understands the idea of how to solve the problem but isn’t actually able to apply the tools needed to solve the problem at hand, for lack of practice. At first glance student A seems to be better of. However, lets see some time into the future. A few years later, at work, a problem pops up that can be solved by finding the maxima or minima of a function. Which one is more likely to think of using what they learned to solve the problem? Student A isn’t used to see the problem outside its well defined structure. You give him a function, you ask for the extrema, and he answers efficiently. Student B understands how the method comes about. He didn’t memorize a set of steps, he understood why they work and why they are needed. He has a mental image of how the problem must be solved. Hence, when such a problem is presented, I believe he is more likely to think of how the solving technique worked. More importantly, if he needs to adapt the problem (say, he wants to find the extreme of the acceleration, so he needs to go all the way to the 4th derivative), he is more likely to be able to adapt the problem.

Foregoing my initial statement, teachers tend to teach both alternatives. First they show alternative 2 to provide understanding and context, and only then they move to alternative 1 which is the goal given. However I have a huge distaste for this approach due to to its poor long term returns. My view is that an extreme (a student A who learns exclusively via Alternative 1 and a student B who learns exclusively via alternative 2), then student B might only have Bs throughtout his studies, while A will appearingly be better by having As initially, but once the gaps and lack of understanding in his learning style start to matter he will sink into Bs, Cs, and so on. That is why I’d rather have understanding than efficiency. As Math with Bad Drawings phrased it, we (teachers) don’t want to put a ceiling on our students cognitive abilities, so we should always focus on Alternative 2. We shouldn’t even start a student on Alternative 1 if he hasn’t understood what is going on. I could go on about this, but at this point I’ll just reccoment you read Math with Bad Drawings’ post, which reiterates how essential it is for us to value understanding over test performance.

Predicting Product Conversion Rates

2017-12-28T00:00:00+00:00

Given the past success of a product, how can we try to predict its future success?

It seems like a harmless question and not too hard. A product which sold 20% of the times it was shown to a costumer has a 20% chance of being bought if we show it to the next costumer, right?

Well, not exactly. In the above example we are using the formula $\frac{S}{V}$, where $S$ is the number of Sales and $V$ the number of times a product was shown (i.e. visits). If a product has no sales yet, it is a bit unfair to say that it has zero chances of being bought.

So how would you solve that problem?

The reasoning we will follow is the following:

Let $X$ be a random variable representing the ‘true’ probability of a product converting. I’m using true as an intrinsic property of the product. We can also view it as the conversion rate of the product as the number of visits goes to infinity.
We will find the pdf $f(\theta)$ of $X$
We will find the expectation of the product converting, i.e. $\mathbb{E}[X]$

All of the steps above are the same as the ones on the previous post on AB Testing. As such I will skip the calculations and go directly to the results:

$X$ will follow a Beta distribution
Letting $S$ be sales and $N = V-S$ be non-sales, i.e. visits without a sale, $f(\theta) = \frac{\Gamma(S+N+2)}{\Gamma(S+1)\Gamma(N+1)} x^S (1-x)^N$
\[\mathbb{E}[X] = \int_0^1{\frac{\Gamma(S+N+2)}{\Gamma(S+1)\Gamma(N+1)} x^S (1-x)^Ndx} = \frac{S+1}{N+2}\]

Hence for a product with $V$ visits and $N$ observations we should expect future clients to buy the product with probability $\frac{S+1}{N+2}$.

Cool, we are done then.

Not yet. Depending on the market / platform where you are selling products, you might have very different conversion rates. Usually they are fairly below 50%. Lets assume for a second that all products we are selling average to a conversion rate of 10%. When a new product comes ($S = 0$ and $N = 0$) then we would predict a future conversion rate of 50%, but intuitively a prediction of 10% should be better.

Yes, that is a problem. How do you fix that?

If we look back at step 2 above we have made subtle assumption: we assumed that we have no prior information about our hidden random variable $X$. If we know nothing about the product, then we are indirectly assuming that $X$ is uniform on $[0,1]$ until we get new information. If you are wondering why assuming no knowledge about $X$ leads us to assuming $X$ is uniform on $[0,1]$, the reason is that the best measure of our knowledge about a system is entropy and the uniform distribution maximizes entropy. If you want to learn more about this I would recommend reading through Shannon’s A Mathematical Theory of Communication. In the meantime I will go back to the question at hand.

We assumed we had no knowledge about $X$ when we fact we do. As such we can use Bayesian probability. The main idea of Bayesian probability is that we have a certain distribution $X_0$ for a hidden random variable, and as time passes we make observations, in our case about sales or lack thereof, and update our estimates $X_t$ as time $t$ moves forward.

And how do we do that?

We will use Bayes’ Theorem, which states that $\mathbb{P}(A|B) = \frac{\mathbb{P}(B|A)}{\mathbb{P}(B)}\mathbb{P}(A)$. In other words the probability of $A$ being true is updated after observation $B$ by the factor $\frac{\mathbb{P}(B|A)}{\mathbb{P}(B)}$. That formula works well for discrete cases, but for continuous cases we get a slightly different formula:

If $f(\theta)$ is our distribution as above and $g(\theta)$ our initial knowledge of the distribution, then our updated knowledge is given by $f(\theta)g(\theta)$ normalized, since a pdf must always integrate to 1. In a no knowledge situation we get $g \equiv 1$ so $f(\theta)g(\theta) = f(\theta)*1 = f(\theta)$ as we concluded before. However in our case we have the knowledge that the average product sells 10% of the time.

How do we translate that into the function $g$ ?

We will have to use our historic data. Assuming we have sold enough different products, then each product has its own conversion rate. As such the set of all our products has its own distribution. If we were to pick one product at random from our inventory then we would be taking a product with conversion rate modeled by that distribution. Hence we can think of adding a new product similar to taking the conversion rate from a product in our inventory. As such we have $g$ and $f$.

So we just multiply the 2 distributions, normalize and we are done?

That will certainly give us an estimation way better than our initial estimate. However there is one problem. Since our inventory might be big, $g$ will probably be defined by a big array and multiplying it by f and then computing $\mathbb{E}[f(\theta)g(\theta)]$ might be computationally expensive.

Can we simplify it?

There is one trick that might work, depending on our product distribution. We can try to model our initial distribution $g$ as a beta distribution. We are getting now exact with our calculations so, depending on the sales data, this might work or not. If it works then we approximate $g$ with a beta distribution with $S_o$ sales and $N_o$ non sales. Then (setting $x = \theta$ for easier typing, and letting $\propto$ mean proportional to) we get

\[\frac{f(x)g(x)}{\int_0^1{f(x)g(x)dx}} \propto\] \[\propto f(x)g(x) \propto\] \[\propto \frac{\Gamma(S+N+2)}{\Gamma(S+1)\Gamma(N+1)} x^S (1-x)^N \frac{\Gamma(S_0+N_0+2)}{\Gamma(S_0+1)\Gamma(N_0+1)} x^{S_0} (1-x)^{N_0} \propto\] \[\propto x^S (1-x)^N x^{S_0} (1-x)^{N_0} \propto x^{S+S_0}(1-x)^{N+N_0}\]

So after normalization $f(x)g(x)$ has pdf

\[\frac{\Gamma(S+S_0+N+N_0+2)}{\Gamma(S+S_0+1)\Gamma(N+N_0+1)} x^{S+S_0} (1-x)^{N+N_0}\]

Which is a Beta distribution with $S + S_0$ sales and $N + N_0$ non-sales. This makes sense intuitively because we can think of $g$ as the distribution of $X$ after $S_0$ sales and $N_0$ non-sales, and $fg$ is the pdf of $X$ updated after we see $S$ more sales and $N$ more non-sales.

But then our estimation for a products future conversion rate becomes $\frac{S+S_0 + 1}{S+S_0 + N + N_0 +2}$ which is both more accurate than the initial formula $\frac{S}{N}$ and equally fast to compute.

Great, so I should always use $\frac{S+S_0 + 1}{S+S_0 + N + N_0 +2}$ instead of $\frac{S}{N}$.

Not really. First of all there’s the previous assumption that $g$ can be modeled via a Beta Distribution. That depends on how rigorous we want to be, but it nonetheless needs to be checked. Secondly this was all about forecasting conversion rates. For analysis related to past performances I would not suggest using this formula since it is disconnected from the truth (i.e. the products actual conversion rate). However, re iterating, any predictive tool related to Conversion Rate should be using this formula instead.

I will summarize with a few pros and cons of each predicted conversion rate. s

Division by zero:

For new products with no observations the formula $\frac{S}{N}$ has no meaning, which can cause problems when doing some bulk analysis i.e. some machine learning on top of it

Cold Start Problem:

The Cold Start Problem is defined by not showing enough products that haven’t been seen a lot yet. If we use the Conversion Rate as a proxy of how much we should show a given product then new products will have $CR = 0$ until their first conversion, which might take a while, and until then they will be barely shown. This does not happen with $\frac{S+S_0 + 1}{S+S_0 + N + N_0 +2}$ and the opposite happens with $\frac{S+1}{N+2}$ which maps new products predicted $CR$ to $\frac{1}{2}$ , which is generally above the average $CR$.

Overvaluing new products:

This is the final statement in the last paragraph. $\frac{S+1}{N+2}$ overshoots the predict $CR$, while $\frac{S+S_0 + 1}{S+S_0 + N + N_0 +2}$ maps new products to the average $CR$, neither boosting nor penalizing them.

Scars

2017-12-27T00:00:00+00:00

I found this lying around Reddit and really enjoyed it.

This story happens in a world where each lie creates a scar on the liar’s body. The bigger the lie, the deeper and larger the mark. One day, you meet someone that only has one scar; it is the biggest one you have ever seen.

He was a real good guy, through and through. Never met anyone quite like him since, never really expect I will either.

People like Joe don’t come around often. Once in a lifetime, maybe, if you’re lucky.

Almost everyone I’ve ever met had the tiny silvered paper cuts of white lies on their fingers. It’s a price of formalities, a camouflage of sorts, as everyone has a few, some deeper cut than others and over the years; opened and reopened time after time. And not just that, but the larger cuts, silvery things on forearms and shins, necks and backs. People lie, it’s just the way of things.

Sometimes the pain is worth the deception, the balancing scale plays out mentally before a person’s mouth opens.

Joining the force was what I wanted. There was a lie I told myself: A Lie I scratched in deep, over and over again. I wanted to change, I wanted my parents to be proud: All lies, tiny scratching lines on my shoulder to create a strange and deceitful pattern that never seemed to heal completely.

In truth, I joined the force because I had nothing left. I joined as a last ditch effort to save myself from rock bottom. Among the elite, surrounded by the brave and the successful, I simply kept my head down. It felt like being a fox, stuck among a pack of wolves. Just being there in the first place felt like deception.

But then, there was instructor Joe.

I had more scars than most, and that earned little trust- but if people were politely cold with me, they were visibly frigid with Joe. See, he didn’t have the traditional marks on his hands, he didn’t have cuts and nicks along his arms, his face or neck: At a quick glance you might have thought him the most honest man alive. In fact, at first people did. A man in his fading thirties without scars?

That’s like a fucking unicorn. They’re more myth and legend than person- yet there he was. Plain as day.

Everyone liked Joe that first week. Everyone wanted to be on good terms with him- I mean, who wouldn’t? In a world of liars and cheats, proof reminded at every twist and turn of the road, who wouldn’t want someone they could trust?

Well, that was before he took of his shirt in the locker-room. Before we all saw the hideous mark that covered half his back. One lie, but the most gruesome thing I’ve ever seen. From his shoulder blade to his ribs, it looked like a crashing comet of red and silvered white. A tiny portion of it just finally healing, a rough tear now recovered again.

It was all the same lie. That’s something you can just tell sometimes, just know it. Usually you can tell how many times too, but whatever the number was which he’d said that aloud, I don’t know.

He rarely spoke to begin with, issuing the orders with a stern smile, instructing as all the rest did. He was positive, encouraging, truthful: But that scar was on everyone’s mind. Deep, dark, and terrible: Someone who could tell a lie like that… Well, there was someone to watch out for. In the end though, it was at the range when things went well and truly sour.

Live-fire runs, we’d done them a thousand times, but that day I guess someone forgot themselves. Maybe they thought too much on what and how and their brain skipped a beat, or maybe they were just careless. Regardless of the reason, a shot fired when it shouldn’t have. Brass spit fire, Air swallowed metal, and lead took its first taste of iron, calcium, iron and dirt.

In that order.

We all stopped, eyes wide and watching that kid fall down real slow. First standing, staring with his hand pulling away- not even scared, just shocked. Red, like deep crimson soaking and spreading, he dropped down to his knees. Still, he wasn’t even there yet, it hadn’t quite processed.

That’s when Joe caught him- and all the shouting erupted. The pandemonium, the first real training turned to action kicking in. Cries for “Medic!” and “KIT! Get the kit!” as people ran for the directions they thought mattered.

I was close enough to know that wasn’t going to make a difference. Center of mass was what we trained for, the reason was straight and forward: Shoot to kill. Eliminate the target and move on.

So I sat there, weapon heavy in my hands as I watched Joe hold this kid, blood pouring out into the dirt like a faucet, and I listened to him repeat the words that cut deep. Over, and over, and over again.

“Hang on, look at me. You’re gonna be alright.”

“You’re gonna be alright.”

GitLaw

2017-12-26T00:00:00+00:00

This post will start with the history of GitHub and Git, founded on the 3rd April 2005, by Linus Torvalds¹. Once upon a time code started to be written. It would be written for websites, operative systems, games, and many other uses. As software got more complex it was harder to manage. It required multiple people working on the same project, and tight cooperation amongst the developers. It all got so complex that it was impossible for anyone to know how it worked in all its detail. As with everything, programming errors were made. When it broke, we first had to understand exactly what was the problem. Then we had to fix the code, but navigating code we don’t understand takes a long while, let alone knowing how to fix it without breaking other parts of the software. This is when Git and other Software Management tools came to help.

Improvements made by Git and other Source Code Management tools
So how does this relate to law?
What is GitLaw?
How is GitLaw achieving what it promises?
That all sounds great, but how do we implement it?
Do you think the above will happen?
Sources & Related Links
Footnotes

Improvements made by Git and other Source Code Management tools

Automatically saving a product’s history

GitHub works by keeping the software’s history using a simple interface. There is a central repository (we’ll call it origin) with the code. A pull request is the same as asking for the latest code version. When I have the code I make commits with comments explaining the changes made. There is a diff operator that allows me to see the changes made between commits (So we can just look at the code changes, instead of having to look at the entire code). When we are done with changing the code we make a push request, which equates to sending our branch to the origin and asking for it to be accepted. Others can see the uploaded code, see the changes committed and make comments on things that are not clear / can be improved. Once all the comments are answered we can finally merge the code.

PS: The above is a simplification of the process, written with clarity for those who never worked with Git over correctness of all processes. For a detailed and fully correct explanation see GitFlow.

Branching of a software to test different hypothesis

I can create 2 branches of a software (i.e. make 2 exact copies of the master code), make different changes and see which branch performs better. After choosing the ‘best’ branch I can make a pull request from it to the master branch, and discard the ‘losing’ branch.

Faster collaboration by working on different parts of the code

Following the methodology above, I can make a pull request and work on component A of the code. You make a pull request and work on component B. We both make a push request and have our requests evaluated independently.

Code reviews became a lot easier with git diff

As described above, after making the necessary code changes and a pull request, other developers will be able to see the changes made and comment on them, while focusing exclusively on the code differences.

Accountability of individual changes

Whenever a change is made, we can trace back who committed the change. That way any doubts regarding the thought process that lead to the code implementation and related questions can be directed easily, making it faster to understand ‘old’ code.

So how does this relate to law?

Law is code, and code is law.

Code is a set of rules that computers follow in order to attain the expected goal. As users require new functionalities and better performance the code gets harder to manage and organize. Code complexity has a cost on its own: it makes it harder for new people to understand and improve. Hence we need source code management tools such as Git.

Law is a set of rules that people follow in order to attain the best for society². As human interactions get more complex the law gets more complex in order to handle all the edge cases. Law complexity has a cost on its own: bureaucracy makes it harder for new businesses to start, transactions to be made and its’ final user, the average citizen, loses track of what he can, can’t and should do. Hence we need Law Management tools such as GitLaw.

What is GitLaw?

GitLaw is a project. It is a project that promises to give transparency and accountability to the legislative branch, while speeding up the process and allowing easier citizens contributions.

How is GitLaw achieving what it promises?

Transparency

Firstly, by having all laws available in a central website, we can easily search for all laws and law books³. Secondly, by having the history of law books available we can trace back changes to those who pushed for them to happen. This would avoid the problem of joining bills so that law changes are made as a bundle, where some small changes are passed without being given proper thought. By allowing for frequent yet shorter bills to be voted independently we bring visibility to the smaller changes.

Accountability

By checking history we can back trace changes done to those who pushed for them. This should make everyone think twice before pushing for a bill change. It adds moral costs (people should think through changing laws more carefully since they are accountable for what they push) while decreasing bureaucratic costs.

Reducing costs of new bills

Following the above, it decreases bureaucratic costs: Managing bills and push requests online reduces management costs, and being able to see law changes as difference files reduces the time wasted to understand what a new bill is doing.

Speeding up the process

Frequently bills are proposed as a list of planned changes. If it passes then a team of lawyers will change law books accordingly. Under GitLaw the government changes the law directly and bills are just the difference files.

Remark: Even though we should be fast to consider and approve new bills, the stability of laws is very important if we want to foster events. Ideally bills would be passed but only became ‘valid’ every x months or so (as a group), so that it wouldn’t keep changing constantly. However this detail and others can and should be more thoroughly though about after implementing GitLaw, since this problem is independent of whether or not we are using GitLaw.

Ease of use

GitHub requires learning, but almost every developer uses it because it quickly pays off for how easy maintaining code becomes. Similarly GitLaw would require a learning phase (probably no more than a few hours), but after that we would be able to see the law at a specific period in time, see who advocated for each bill, check the discussions had before passing each bill to know how well it was though about.

That all sounds great, but how do we implement it?

This is the tricky part (as with everything). Personally I have no doubts about the usefulness of such system, but for any change to go through there must be people pushing for it to happen. Society benefits as a whole (and a lot), but there is no one who benefits enough to push for it. In order to bring interest to it either we find a great bunch of philanthropists willing to put in the hours (ideal but usually unlikely given how many other problems we have to work on), or we need to find either public funding or a sustainable business model to implement this.

Step 1: Developing a law book filetype ( LawBook )

The most similar occurrence of this is Fountain, as mentioned in Abe Voelker’s blog. The idea is simple: instead of having PDFs, Microsoft Word / Google Docs or another general purpose filetype to edit law books, we should have a specific one designed for exclusively for laws. The format of law books does not differ significantly from each other. I would compare it to designing a syntax for Screenplays similar to what Fountain achieved.

The filetype should have metadata such as creation date, author, owner and others included. In the file itself we should be able to define sections, sub sections,… and reference each other. This part needs to be better thought about, but ideally we would need at least a lawyer involved (to enlighten us about how to generalize a law book’s structure) and a programmer (ideally someone who is familiar with parsing documents, to avoid possible problems). Maybe a variation of markdown would be ideal.

Remark: It is common to define the meaning of certain words to avoid ambiguity, such as in the RFC2119. We could just have all such defined words link to their definition, decluttering space.

Ideally we would end with a page like Fountain’s Syntax Page, which explain how to write with this syntax.

Step 2: Developing a web / local editor

The LawBook we defined above could directly be edited via a text editor, but we want to make it easier and more intuitive to use. As such we need a text editor. It would need to allow:

Add sections / subsections and others.
References to other sections
Add word definitions (or link to word definitions in other documents)
Warn about broken links, since we might point to a document and after a while that document is changed, rendering the link invalid.
(Optional) Reference to other law books.
etc

It should be as plain to use as possible. Keeping it simple will make it easier to use. That’s why I personally prefer a web based editor, but this is up to debate.

Step 3: LawBook to pdf

Essential for easy printing / sending rulebooks to people. Shouldn’t be hard to implement. Since we are aiming this pdfs for possible printing then we need to think carefully how to resolve both internal links and links to other LawBooks.

Step 4: Setup with Git

Setup a repository, create a few LawBooks and make changes, see how it behaves.

Step 5: Build GitLaw

Probably the hardest part. GitLaw would be the effective GitHub for Law, a browser based editor where people could see the central repository branches, diffs, comments, push requests (i.e. bills) and everything else. This also needs to be well though of before implementing. A good starting point is to analyze GitHub functionality and see where it has unneeded and/or missing features for the GitLaw needs.

Step 6: Move LawBooks to GitLaw

All publicly available LawBooks can be moved to GitLaw, without the need for legal permission (after all LawBooks are already publicly available). Having an easy to read online LawBook reader is crucial to bringing people’s attention to the tool. If it is easier to use than tracking all the law books scattered through different institutions then it will be used.

Step 7: Give repositories ownership to the respective institutions

Allow institutions to alter the LawBooks themselves. We need good tutorials on how to use the tools we provide, but this project should be straightforward.

Do you think the above will happen?

Yes. Steps 1 to 3 shouldn’t be too hard with part time investment. Step 4 is incredibly hard and probably will require full time investment. Maybe discussing the possibility with GitHub itself would make it easier. Maybe not. Step 5 is time consuming but isn’t hard. Step 6 is the easiest one if we advertise our goal during the process. All of the above should be made into an MVP, and then improve on that by iterating over the various steps. By starting as a lookup tool with history (for lawyers to see the current laws and nothing else), it should get some traction.

Abe Voelker’s blog - GitLaw: GitHub for Laws and Legal Documents - a Tourniquet for American Liberty

Being considered by the French Government - forum.etalab.gouv.fr

EU - European Parliament to share amendment web tool as open source

Footnotes

¹ Git - How it actually came to be instead of the short tale I tell.

² If this is the one and only goal of having laws is debatable, but lets assume this is true for now. It shouldn’t be too far from this anyway.

³ By law book I mean a document where a set of laws are written. Law book sounds weird, but I didn’t find a better term. Tell me if you know the actual name for those.

Nash's Equilibrium

2017-12-22T00:00:00+00:00

Lets play a game. It is called Rock Paper Scissors. After each round the winner receives 3£¹ from the loser. However I have an handicap. I can never ever play Scissors. How much should you pay to play this game?

Somewhere between 0£ and 3£ ?

True. If you payed 0£ you would not be expecting to lose money and if you paid 3£ you would certainly not make a profit, so you should pay something in between those values. However we can do better than that. We can find the specific value it is fair to pay.

How?

Lets try to understand the game better. I can only play Rock or Paper. Hence the the possible outcomes for you are:

	Rock	Paper
Paper	3£	0£
Scissors	-3£	3£
Rock	0£	-3£

Since playing Rock is always worse for you than playing Paper we can assume you never play it, so the possible outcomes become:

	Rock	Paper
Paper	3£	0£
Scissors	-3£	3£

If you always play Paper you never lose money and no matter what we play you will never win more than 3£ , so your initial guess of paying something between 0£ and 3£ makes sense. If you always played Paper then I would always play Paper and you would win no money. However you could change your tactic to playing Scissors in order to win the 3£, but then you will be at risk of losing 3£ if I play Rock.

How does that help us reach the fair value?

In practice in every play we will choose each move independently and with certain probabilities. For instance, I will choose Rock with probability $p$ and Paper with probability $1-p$ (remember that the sum of the probabilities of all of our moves must add up to $1$). You will choose to play Scissors with a probability $q$ and Paper with probability $1-q$. As such the probabilities of each move are:

	Rock	Paper
Paper	$$p(1-q)$$	$$(1-p)(1-q)$$
Scissors	$$pq$$	$$(1-p)q$$

Let $L(p,q)$ be how much money you expect to make given the above probabilities. Then

\[L(p,q) =\] \[= [pq(-3)] + [(1-p)q(3)] + [p(1-q)(3)] + [(1-p)(1-q)(0)] =\] \[= -3pq + 3(1-p)q + 3p(1-q) =\] \[= 3(p + q - 3pq)\]

Cool. Now we have a function giving us how much you expect to get after a game. I will try to choose $p$ so that $L$ is minimal, and you will try to choose $q$ so that $L$ is maximal. Nash’s work (and the reason for this post’s title) was about proving that this equilibrium always exists for non-deterministic games (and this is called Nash’s Equilibrium)

So how do we choose $p$ and $q$ ?

We have to go back to finding maxima and minima of functions (if your high school math is fresh you should probably skip this section). We have to choose $p$ in the interval [0,1]. Therefore either there is a local maxima somewhere in the middle or the maximum is attained at the endpoints. To find interior maxima we will use derivatives: If $(p_0,q_0)$ is a maxima or minima, then the derivative of $L$ with respect to $p$ vanishes (i.e. equals zero) at $p_0$. Doing the actual computations we get:

\[0 = \frac{\partial }{\partial p}(L) =\] \[= \frac{\partial }{\partial p}(3(p+q-3pq)) =\] \[= 3(1 - 3q) \implies q = \frac{1}{3}\]

Similarly,

\[0 = \frac{\partial }{\partial q}(L) \implies q = \frac{1}{3}\]

Hence our local maxima / minima can be found when $p \in \{ 0,\frac{1}{3},1 \}$ and $q \in \{ 0,\frac{1}{3},1 \}$.

Visually the expected returns for you (i.e. $L(p,q)$ ) are:

$$q \setminus p$$	$$0$$	$$\frac{1}{3}$$	$$1$$
$$0$$	$$0$$(A)	$$1$$(B)	$$3$$(C)
$$\frac{1}{3}$$	$$1$$(D)	$$1$$(E)	$$1$$(F)
$$1$$	$$3$$(G)	$$1$$(H)	$$-3$$(I)

I can choose the value of $p$, which amounts to choosing the column we are in. Since I want to minimize your returns (i.e. minimize $L$) then I am happy if, for any line you choose, I choose the column with minimal $L$. Hence my choice will end up being one of (A),(D),(E),(F),(I). (A) and (I) because they are the minimum in their respective rows, and (D),(E),(F) because they are all equal in their row.

Similarly you get to choose $q$ i.e. you choose the row. In order to maximize $L$, you will only be happy with (C) and (G) since they are the maximum in their respective columns, or (B),(E),(H), for a final choice in the set (C),(B),(E),(H),(G).

Looking at the 2 sets we chose: $\{A,D,E,F,I \}$ and $\{ C,B,E,H,G \}$ we notice that they only intersect in E (Nash proved that when following this method there will always be an intersection. If you want to know how I would recommend reading his paper, though it is way more advanced than this post). As such when we both play optimally we will be in cell (E).

I lost track. How does this relate to the original problem?

We saw that If we both play optimally then we will end up in cell (E), where $p = \frac{1}{3} = q$ . Since $p$ is my probability of playing Rock and $q$ your probability of playing Scissors, then we found the optimal strategy for both of us.

Also, for those probabilities, $L(\frac{1}{3},\frac{1}{3}) = 1$ so you will have an average return of 1£.

Therefore, going back to the original question, paying 1£ would make it a fair game. Anything less than that and you will most likely profit, anything more than that and you will most likely lose money.

_{¹ You are probably wondering why I choose 3£ instead of 1£ . During the calculations we end up having to divide that value by 3, so this way I have to work with less fractions.}

Michellin

2017-11-09T00:00:00+00:00

For any of you who have a car, you might have had the experience of getting a tire blown and have to pay for it. This is not a big problem (The automobile industry needs to profit and you need a tire, so it seems like a win win situation). There is a problem with incentives whoever. Michellin profits whenever you need a replacement tire. The more often you need a replacement tire the more they get paid. Hence, there is no reason for them to improve the quality of their tires.

That is problematic. How can we solve it?

There is a solution currently being tested by Michellin. Instead of paying per tire, you are being paid per kilometer driven. This is being possible due to the diminishing costs of technology. In particular, Michellin is equipping tires with sensors that measure the distance traveled, temperature, weariness and many others. This way they are able to charge for the number of kilometers driven. Not only that but they are kept up to date about the tires weariness and can inform their clients about needed tire changes if needed.

Great, so we have a solution.

Lets not jump to conclusions too fast. We need to consider the incentives in case A (pay per tire) and case B (pay per kilometer)

Case A:

Since we are paying per tire Michellin has no incentive to improve the quality of the tires. Clients have to be careful in their driving in order to avoid unneeded costs. By default costs associated with tire replacement are paid by the Client.

Case B:

Since we are paying per mile it is in Michellin’s best interest to minimize their cost per km driven, hence making better tires. By default costs associated with tire replacement are paid by Michellin. Michellin is also able to call clients for a tire change if their sensors raise any alarms. Clients do not need to worry as much about wearing out tires.

But you still agree that Case B is best, right?

Definitely yes, but I wanted to bring attention to the incentives in play. By Michellin’s own initiative they were able to change the company incentives so that investing in durable tires is in the company’s best interest. Aligning society’s interests with company’s interests is more powerful than legislation or company claims to do so when research and other company sectors might not be transparent to the public. There are many industries where similar changes can be made, so do share any ideas that come to your mind.

Kudos to Michellin for doing this out of their own initiative. I don’t believe this option (pay per kilometer) is available to everyday single car customers, but this solution is being rolled out already to fleet owners (be it trucks, airplanes or other wheeled vehicles).

Differential Cryptography

2017-10-06T00:00:00+00:00

Apparently Apple is taking steps to protect their users privacy by using Differential Privacy.

So what is differential privacy?

I wondered the same after reading the article, since they mention its benefits a lot without explaining what it is. After reading about it for a bit in Wikipedia under the title Differential Privacy, I understood the goal of differential privacy to be messing with the data of individuals (making it useless for individual targeting purposes) while preserving its statistical properties as a whole. In essence is a way of extracting value from knowledge of millions of individuals without invading the privacy of a single individual.

That sounds great, but how can you do that?

The best example I was able to come up with: say that you want to survey people in order to find how many people smoke weed. Since it is illegal to sell weed and there’s a big discussion about it, people might not feel comfortable disclosing whether they smoke weed or not. Instead, lets put a little math into this:

Let each person flip a coin. If it lands heads then the person answers truthfully, otherwise it answers at random (with another coin flip). Say that $q$ of the population actually smokes weed. Then through this method we expect to have $p = \frac{q}{2} + \frac{1}{4}$ of the population to answer yes. Rearranging the equation we get $q =2p-\frac{1}{2}$. Hence we can retrieve (with some statistical noise) a rough estimate of the population that actually smokes weed. However, if we were to go to a single individual and claim that they smoke weed, we couldn’t since we couldn’t be sure whether he actually smoked weed or wether it was chance that made him answer yes.

So the use of differential privacy is to protect wrongdoers?

No. First of all wrongdoing is subjective, but I won’t go there now. I’ll try to keep this post more math and tech related. Secondly, Machine Learning has made the need for Differential Privacy increase. Here’s an example: Say we want to build software capable of diagnosing illnesses. AI needs a lot of data for that, in particular medical records of thousands of people. We can hide the name of the patient and any ID associated with him before releasing the data to the AI corporation, but this is not a risk free action. From intel such as height, body weight, race, gender and by joining any medical condition we could find the original individual to whom the data belonged to, and maybe learn about some condition that would preferably be kept private. By messing with the data a bit, we can increase individual privacy at the cost of the AI’s accuracy. This is the big trade off in Differential Privacy¹.

So using Differential Privacy is awesome and we should admire Apple for using it :D

Yes and No. Using Differential Privacy is great, but Apple didn’t release any info on how they are using it. They might be using it on a tiny insignificant part and claiming it to be a big thing, or they might be actually doing a great job. It’s hard to know when companies keep so much information to themselves, which makes sense financially but for outsiders it makes us stand stronger behind initiatives like the Open Source Software Movement.

¹ Note that I haven’t given a specific method of jumbling the data, that needs proper and careful thought which I might give at a later time. Probably not though.

How to AB Test

2017-10-05T00:00:00+00:00

This article is mildly related to Google’s Multi Armed Bandit experiments.

What are AB Tests?

Suppose that you have a problem (say you want to be the best CoD player) and you have many possible solutions to your problem (namely different play styles, from the most aggressive to the most defensive, using different weapons, and many more alternatives). You have some ideas as to what are the best tactics, but you’re not sure which one would fit you best (in this case best means the ratio between winning and losing matches).

What AB Tests propose is that you try out the different options for a length of time. After that time you will have played each option $i$ $n_i$ times and have won $r_i$ times. As such $\frac{n_i}{r_i}$ will give you an approximation of the win ratio for each alternative, and you can make an informed decision on what weapons to use, your play style, etc.

Wouldn’t it take ages to test all the different options?

Yes! That is why the multi armed bandits come into play. Say that you have a certain play style that you think is the best. You define it as your champion and play with it most of the time. However whenever there’s an alternative you think might outplay your current option you bring it into the alternatives pool, and you play with it occasionally. Then you can start ruling out alternatives who have too few wins and betting more on the most promising alternatives, instead of waiting for a fixed amount of time / trials (and possibly sacrificing your long term win rates). You need to be careful not too discard alternatives too soon (just because you lost 3 matches in a row doesn’t mean it’s a bad alternative, maybe it was just bad luck. Some people call this statistical significance), and similarly not to bet exclusively on the higher end alternatives.

So how do you know which ones to use?

There are different suggestions for solving this problem. I would be interested in seeing AB Tests on AB Testing Techniques. However in this post I am going to provide one alternative that I think is best. Do comment below better alternatives / points I’ve missed.

That’s all good, but I haven’t heard your solution yet.

Ok, say you have $m$ alternatives of play and you are about to play a match. In the past you have played $n_i$ matches with option $i$, of which you won $r_i$ matches. My current goal is to maximize the percentage of wins over a long time period. So I’ll make this formal and I’ll say I want to maximize my expected wins over the next $k$ matches.

Warning: I’m going to get more technical from here onwards.

We can then define our problem as one of finding the following function:

\[C(n,r,k) : (\mathbb{N}^{+})^{m} \times (\mathbb{N}^{+})^{m} \times \mathbb{N}^{+} \mapsto I_m\]

Which maps the vector $n = (n_1,n_2,...,n_m)$ and $r = (r_1,r_2,...,r_m)$ of observations and wins and the integer $k$ to the integer $i$ in $I_m = {1,2,...,m}$ which is the best choice of alternative to play in order to maximise the long term win rate. In order to calculate this we need another function

\[V(n,r,k) : (\mathbb{N}^{+})^{m} \times (\mathbb{N}^{+})^{m} \times \mathbb{N}^{+} \mapsto \mathbb{R}\]

which maps the same knowledge to the expected number of wins in the next k matches assuming optimal choices. Now we do some maths:

How likely am I to win if I choose to play with alternative $i$ ?

So you’ve had $n_i$ observations so far and $r_i$ wins. Let $\theta$ be the underlying likelihood of winning. Since we know $n_i$ and $r_i$ then we can estimate the pdf¹ of $\theta$ by using Bayes Theorem. Letting $a=n_i$, $b=r_i$ we first compute:

\[\mathbb{P}( X=b | \theta) = C^a_b \theta^b (1-\theta)^{a-b}\]

where $X$ is the sum of $a$ Bernoulli variables with probability $\theta$ and $C^a_b$ is the number of ways of choosing b elements out of $a$ elements. From this we can get the pdf of the underlying $\theta$ by normalizing the pdf:

\[f(\theta) = \frac{C^a_b \theta^b (1-\theta)^{a-b}}{\int_{0}^{1}C^a_b t^b (1-t)^{a-b}dt} = \frac{\theta^b (1-\theta)^{a-b}}{\int_{0}^{1} t^b (1-t)^{a-b}dt}\]

Since $a$ , $b$ are non-negative integers, we can use the fact that

\[\int_{0}^{1} t^a (1-t)^{b}dt = \frac{a! b!}{(a+b+1)!}\]

to simplify $f(\theta)$ to

\[f(\theta) = \frac{\theta^b (1-\theta)^{a-b}}{\frac{a! (a-b)!}{(a+1)!}} = \frac{(a+1)! \theta^b (1-\theta)^{a-b}}{b! (a-b)!}\]

Finally we can get the probability of success in a single sale by integrating the chances of success over the pdf:

\[\int_{0}^{1} \theta f(\theta) d\theta =\] \[\int_{0}^{1} \theta\frac{(a+1)! \theta^b (1-\theta)^{a-b}}{a! (a-b)!} d\theta =\] \[= \frac{(a+1)!}{b! (a-b)!} \int_{0}^{1}\theta^{b+1} (1-\theta)^{a-b}d\theta =\] \[= \frac{(a+1)!}{b! (a-b)!} \times \frac{(b+1)!(a-b)!}{((a-b)+(b+1)+1)!} =\] \[= \frac{(a+1)!}{b! (a-b)!} \times \frac{(b+1)!(a-b)!}{(a+2)!} =\] \[= \frac{(a+1)!(b+1)!(a-b)!}{b!(a-b)!(a+2)!} =\] \[= \frac{(a+1)!}{(a+2)!} \times \frac{(b+1)!}{b!} \times \frac{(a-b)!}{(a-b)!} =\] \[= \frac{1}{a+2} \times (b+1) \times 1 =\] \[= \frac{b+1}{a+2}\]

Awesomeness, all of the above simplifies to a really neat expression. If we played $n_i$ matches with $r_i$ wins then the chances of winning the next match is $\frac{r_i + 1}{n_i +2}$ . As a bonus that expression works even if $n_i = r_i = 0$, since in that case we are assuming the pdf of $\theta$ to be the entropy maximizing (and hence knowledge minimizing) uniform distribution, in which case there’s a $\frac{1}{2}$ chance of winning. On second thought, that is to be expected since the pdf and integral above only require $n_i$ and $r_i$ not to be negative, so there’s that.

Yay, now we know that the chances of success by choosing option $i$. How does this help us in making long term choices?

We have found the solutions of $C$ and $V$ for $k=1$, namely:

\[C(n,r,1) = argmax_i \{ \frac{r_i + 1}{n_i + 1} \}\] \[V(n,r,1) = C(n,r,1)\]

Now we need to solve this same problem for $k>1$. If, for a given $k$, we choose option $i$ , then our expected number of wins is given by

\[\mathbb{P}(win) (1+V(n+e_i,r+e_i,k-1)) + \mathbb{P}(loss) V(n+e_i,r,k-1) =\] \[= \frac{r_i + 1}{n_i + 2} (1+V(n+e_i,r+e_i,k-1)) + \frac{n_i-r_i+1}{n_i+2} V(n+e_i,r,k-1)\]

As such we only need to find the $argmax_i$ of the expression above to find $C(n,r,k)$, and use that to find $V(n,r,k)$. Hence we have found a recursive formula for this, which can be solved somewhat efficiently using dynamic programming³ since there will be many values computed repeatedly.

Great, now what?

Well, I still need to implement this, probably in a python notebook that I’ll leave available somewhere (only then can I check how feasible it is to compute this result). If it turns out to be computationally unfeasible (due to huge $k$’s) then I’ll try to tackle it in a slightly different way:

Instead of choosing one alternative at a time, I can choose them in a bulk of size $b$, and then decide how many of that bulk should be used in each alternative (effectively deciding a percentage to be used in each alternative). By having large bulk sizes I could avoid having absurd recursion depths of millions of function calls just to compute this value, instead going for recursion depths of hundreds with bulk sizes of tens of thousands (we can play with this numbers after implementation).

EDIT: I’ve found this paper online about Batch Size implementations of this problem. I might write a short summary of it after having read it properly.

Footnotes

¹ pdf : Probability Density Function and its’ Wikipedia page

² $e_i$ : Unit vector along the $i$th dimension, i.e. $(0,0,...,0,1,0,...,0)$

³ I never understood why they call it dynamic programming. There’s nothing dynamic about this technique.

The Mythical Man Month

2017-05-15T00:00:00+00:00

It’s rare to find such a short book with so many useful insights. This insights are based on the work experience of Fred Brooks. In summary he mentions:

The mythical man-month
No silver bullet
The second-system effect
The tendency towards irreducible number of errors
Progress Tracking
Conceptual Integrity
Formal documents
Project estimation
Communication
Specialized tools

The mythical man-month

The core idea in the books states that many project managers (wrongly) use Man-Months as a developer’s productivity measurement unit, so that for example 4 developers do in 2 months what 2 developers do in 4 months. This fails to take into account different group mechanics: 2 great developers might not work well together or a few very specialized developers might achieve no work alone, but when together produce exponentially more work. As the number of workers increases we get more communication pairs (to be exactly $\frac{n(n-1)}{2}$ channels , so with as few as 10 developers we have 45 channels of communication ). Also there’s another problem: most people think about the man-month unit of measurement when their projects are running late. However consider a scenario when you have 4 people, you have 2 months to finish a project but you need 12 man-months (Assuming this is a viable measurement unit). So you need to hire 2 people in order to complete the required 12 man-months. However the new developers need training in order to start working on the project. If they each need a person to introduce them and they will be learning in their first month, then we will only have 2 developers working in the first month and 6 in the second, for a total of 8 man-months. Not optimal.

No silver bullet

Brooks insists that there is no technology or discovery in software that will lead to a tenfold increase in productivity over the span of a decade (this in contrast to Moore’s law about Hardware productivity).

The second-system effect

This revolves around the tendency to over-engineer the second system that is built for any purpose, for the reason that we often try to include in the second system all of the first system capabilities and every addition we didn’t add in the first system due to time constraints.

The tendency towards irreducible number of errors

Brooks states that as systems get complex, there is a lower bound to how many errors the system has. Any attempt at fixing known errors will introduce even more errors. As the fable goes,

little bugs in the code.
little bugs.
Take one down, patch it around.
little bugs in the code.

Progress Tracking

Brooks wrote “Question: How does a large software project get to be one year late? Answer: One day at a time!” Incremental slippages on many fronts eventually accumulate to produce a large overall delay. Continued attention to meeting small individual milestones is required at each level of management.

Conceptual Integrity

Any software should be made user friendly and it pays off to have an architect focusing on understanding what/how the user will be using the program. Everyone in the team should understand the architects’ (i.e. the user’s) point of view and good features might not be included if they make the program harder to learn.

Formal documents

Every project manager should create a small core set of formal documents defining the project objectives, how they are to be achieved, who is going to achieve them, when they are going to be achieved, and how much they are going to cost. These documents may also reveal inconsistencies that are otherwise hard to see.

Project estimation

Project times must take into account the fact that most customer oriented projects take thrice as long as in-house projects and that there will be time spent on administrative tasks (meetings, discussing project ideas,…). A good rule of thumb is to divide the time spent coding in 12 time blocks:

$4$ blocks spent designing the app and it’s structure
$2$ blocks spent coding
$3$ blocks spent testing each component of the app (and debugging if necessary)
$3$ blocks spent testing the system as a whole (and, again, debugging if necessary)

Communication

Everyone working on a project should be easily reachable and the project’s goal should always be made clear. Instead of assuming something, implementers should ask and clarify anything about a feature they are implementing, before proceeding with an assumption that might very well be completely incorrect. The architect(s) are responsible for formulating a group picture of the project and communicating it to others.

Specialized tools

Instead of every programmer having his own special set of tools, each team should have a designated tool-maker who may create tools that are highly customized for the job that team is doing, e.g., a code generator tool that creates code based on a specification. In addition, system-wide tools should be built by a common tools team, overseen by the project manager.

Medium

2017-04-04T00:00:00+00:00

I’ve recently started spending hours daily on a single website: Medium . I see it as a news website where anyone can publish stories, but due to Medium’s underlying magic the post quality is awesome (Otherwise I wouldn’t have swapped The Economist, Financial Times and others for Medium). Here are some posts to get you started:

Familiarity Bias - Just recently discovered this bias (and it makes total sense), Definitely worth reading.
The Natural Order of Money and Why Abstract Currencies Fail - I don’t agree with his conclusion, not only but mostly because he his comparing abstract currencies to Gold/physical currencies instead of Euros/Pounds/Dollars/fiat money. Still, his post was surprisingly informative.

48 Tiny Habits That Will Make You Awesome - I’m surprised at how much little changes can accomplish.
How Bitcoin Works - Nothing is better than reading the original paper (which he links to in his article), but this is a softer introduction if you are not familiar with the concepts involved.
33 lessons from the trenches of building a million dollar company - A list of useful ‘lessons’ to consider, even though he doesn’t go in-depth about any of them.
Here are 250 Ivy League courses you can take online right now for free - Self-explanatory

The above are all oriented to what I enjoy reading about. Medium has various topics which I did not cover. So read away, follow anyone whose articles you found interesting and subscribe to Medium to support their work.

Maths' Ceiling

2017-03-17T00:00:00+00:00

This is entirely based on a post from one of my favorite blogs: Math With Bad Drawings. Read the original post if you wanna see really cool drawings. The post starts with a simple question (and the author provides a lovely answer):

Does each person have a math ceiling? (or a cognitive ceiling of any kind)

Answers usually fall in one of 2 categories. The optimist claims that there is no such thing and everything is achievable by enough work and determination. Some might get it faster than others, but there is no limit. The pessimist claims the opposite: No matter how much one works there are things beyond some people’s comprehension (I’ve heard this mostly from teachers and tutors who taught hundreds of students, with astonishingly disparate performances). Some middle ground can be achieved (someone claiming that there’s no math ceiling as long as you find the right circumstances, including the right teacher, the right mood and motivation to work,…). Still, these are the 2 main campuses of thought.

And then comes Math with Bad Drawings. They start with a beautiful analogy: Say I’ve bought a futon for my room. As I was getting it in, I saw a small metal piece come off. I’m not sure where from, and after looking around and everything seeming to be in place, I have no choice but to take it into my room as planed. All is fine. A week in and the futon starts bending down in the middle. As more time progresses the bending gets worse and worse, to the point of me not being able to sit in it anymore without sliding to the center (and everyone else sitting down too, into a awkward cluster of people).

And so it is with maths.

When we are learning maths we are supposed to follow the rhythm of the class. When we fail to do so we take shortcuts in order to do as everyone else, prioritizing implementation over understanding. We might be able to differentiate equations to find maxima/minima, but that’s useless unless we understand that the derivative is nothing but the slope of the curve at that point and as such the zeros of the derivative give us the maxima/minima. This is obvious to the half of the class that never took shortcuts in previous years and tried to understood everything. For the other half, they’ll feel like they’re just not capable of understanding the concepts involved, and going back to relearn the misunderstood concepts properly is a lot more work than learning them correctly the first time around. As Math With Bad Drawings said, a student who doesn’t understand a topic is a student with an expiration date on what he’ll be able to do.

My final remark to this is the following: it is easier to be an A* student for the entirety of your studies than to be a B or C student for the same period. So put on the effort and climb up the ladder: never take a “You’ll understand it once you do it enough times” for an answer, first understand it and them practice implementing it.

PS: I kind of feel really bad for taking the punchline “And so it is with maths” word by word from Math With Bad Drawings, but it is a brilliant punchline, I couldn’t deprive people of that. Sorry Math With Bad Drawings.

OxfordHack

2016-11-21T00:00:00+00:00

This weekend there was a 24h Hackathon which brought together people from many different countries: OxfordHack. It was a rather unexpected event: Saturday I went to the Maths Institute to revise, but every room was taken by people coding. Completely unaware of what was going on, I saw a welcome stall and asked them what it was all about. Turns out some companies funded this hackathon for 24h, and they had an interesting list of activities scheduled (namely Werewolves or, as I prefer to call it, Mafia). I asked them “Can I join?” and got a successful “Yes”, which was surprising given that in their website Applications were closed. As they were telling me to find a team, Manish, who was passing behind me, said he was looking for people to join his team, and so that was sorted. He already had a project in mind, and goes as follows:

The Netherlands (Manish is Dutch) will be having elections next March, and the candidates will be officially announced on the 17th Dec. As expected those who want to run for president already have started their campaigns and are well known, even before their official announcement for presidency. Turns out that the current most popular candidate, Mark Rutte, is very similar to Trump. Following the belief that a significant amount of the votes come from misinformation and not genuine willingness to support Mark Rutte, Manish is invested in raising awareness of the urgent topics that need to be discussed instead of letting the election being decided by catchphrases such as “Take back control” and “Make America Great Again”. Whether this sentiment reflects the reality of the situation or not, that’s a different topic which I will not approach here. However the idea that a informed society makes wiser decisions is hard to question, and in support of that we are trying to bring hot topics to the attention of the public. As this was an hackathon, the project consisted of gamifying aspects of political debates, where knowledge of both facts and candidate’s policies are tested. Would you like to be put to the test?

We have designed the structure of the website (without coding), with a few topics being open in the sense that we have contradicting ideas regarding certain aspects of the app, but nothing that would prevent us from developing a single app together. We started building it using the ionic framework, but much more needs to be done (and fast if we want to release it before the Dutch elections).

Other than the project we played werewolves (a.k.a. mafia) during the night (I managed to get myself killed in both games during the first round despite being a victim) and there was one interesting challenge I watched: competitors were provided with a manual of html and css code, but no Internet access. Then we were shown Gmail’s front page and we had to recreate it in 20mins from scratch. Whoever gets the most similar front page wins. A couple of participants got my vote for doing the “404 error” page, but sadly/expectedly they didn’t win.

I ended up sleeping in the Maths Institute (as many others did), and when I woke up I started working on what I went to the Maths Department for: actual academic work. That was the end of the hackathon for me, but the project still lives. Let’s see what we make of it.

Egg

2016-03-10T00:00:00+00:00

I found this text lying around on this website. The story told regards a man who dies and finds himself in the afterlife with a divine entity.

You were on your way home when you died.

It was a car accident. Nothing particularly remarkable, but fatal nonetheless. You left behind a wife and two children. It was a painless death. The EMTs tried their best to save you, but to no avail. Your body was so utterly shattered you were better off, trust me.

And that’s when you met me.

“What… what happened?” You asked. “Where am I?”

“You died,” I said, matter-of-factly. No point in mincing words.

“There was a… a truck and it was skidding…”

“Yup,” I said.

“I… I died?”

“Yup. But don’t feel bad about it. Everyone dies,” I said.

You looked around. There was nothingness. Just you and me. “What is this place?” You asked. “Is this the afterlife?”

“More or less,” I said.

“Are you god?” You asked.

“Yup,” I replied. “I’m God.”

“My kids… my wife,” you said.

“What about them?”

“Will they be all right?”

“That’s what I like to see,” I said. “You just died and your main concern is for your family. That’s good stuff right there.”

You looked at me with fascination. To you, I didn’t look like God. I just looked like some man. Or possibly a woman. Some vague authority figure, maybe. More of a grammar school teacher than the almighty.

“Don’t worry,” I said. “They’ll be fine. Your kids will remember you as perfect in every way. They didn’t have time to grow contempt for you. Your wife will cry on the outside, but will be secretly relieved. To be fair, your marriage was falling apart. If it’s any consolation, she’ll feel very guilty for feeling relieved.”

“Oh,” you said. “So what happens now? Do I go to heaven or hell or something?”

“Neither,” I said. “You’ll be reincarnated.”

“Ah,” you said. “So the Hindus were right,”

“All religions are right in their own way,” I said. “Walk with me.”

You followed along as we strode through the void. “Where are we going?”

“Nowhere in particular,” I said. “It’s just nice to walk while we talk.”

“So what’s the point, then?” You asked. “When I get reborn, I’ll just be a blank slate, right? A baby. So all my experiences and everything I did in this life won’t matter.”

“Not so!” I said. “You have within you all the knowledge and experiences of all your past lives. You just don’t remember them right now.”

I stopped walking and took you by the shoulders. “Your soul is more magnificent, beautiful, and gigantic than you can possibly imagine. A human mind can only contain a tiny fraction of what you are. It’s like sticking your finger in a glass of water to see if it’s hot or cold. You put a tiny part of yourself into the vessel, and when you bring it back out, you’ve gained all the experiences it had.“

“You’ve been in a human for the last 48 years, so you haven’t stretched out yet and felt the rest of your immense consciousness. If we hung out here for long enough, you’d start remembering everything. But there’s no point to doing that between each life.”

“How many times have I been reincarnated, then?”

“Oh lots. Lots and lots. An in to lots of different lives.” I said. “This time around, you’ll be a Chinese peasant girl in 540 AD.”

“Wait, what?” You stammered. “You’re sending me back in time?”

“Well, I guess technically. Time, as you know it, only exists in your universe. Things are different where I come from.”

“Where you come from?” You said.

“Oh sure,” I explained “I come from somewhere. Somewhere else. And there are others like me. I know you’ll want to know what it’s like there, but honestly you wouldn’t understand.”

“Oh,” you said, a little let down. “But wait. If I get reincarnated to other places in time, I could have interacted with myself at some point.”

“Sure. Happens all the time. And with both lives only aware of their own lifespan you don’t even know it’s happening.”

“So what’s the point of it all?”

“Seriously?” I asked. “Seriously? You’re asking me for the meaning of life? Isn’t that a little stereotypical?”

“Well it’s a reasonable question,” you persisted.

I looked you in the eye. “The meaning of life, the reason I made this whole universe, is for you to mature.”

“You mean mankind? You want us to mature?”

“No, just you. I made this whole universe for you. With each new life you grow and mature and become a larger and greater intellect.”

“Just me? What about everyone else?”

“There is no one else,” I said. “In this universe, there’s just you and me.”

You stared blankly at me. “But all the people on earth…”

“All you. Different incarnations of you.”

“Wait. I’m everyone!?”

“Now you’re getting it,” I said, with a congratulatory slap on the back.

“I’m every human being who ever lived?”

“Or who will ever live, yes.”

“I’m Abraham Lincoln?”

“And you’re John Wilkes Booth, too,” I added.

“I’m Hitler?” You said, appalled.

“And you’re the millions he killed.”

“I’m Jesus?”

“And you’re everyone who followed him.”

You fell silent.

“Every time you victimized someone,” I said, “you were victimizing yourself. Every act of kindness you’ve done, you’ve done to yourself. Every happy and sad moment ever experienced by any human was, or will be, experienced by you.”

You thought for a long time.

“Why?” You asked me. “Why do all this?”

“Because someday, you will become like me. Because that’s what you are. You’re one of my kind. You’re my child.”

“Whoa,” you said, incredulous. “You mean I’m a god?”

“No. Not yet. You’re a fetus. You’re still growing. Once you’ve lived every human life throughout all time, you will have grown enough to be born.”

“So the whole universe,” you said, “it’s just…”

“An egg.” I answered. “Now it’s time for you to move on to your next life.”

And I sent you on your way.

G-Research Competition

2015-10-24T00:00:00+00:00

Today Me, Matt and Aiken went for a high-frequency stock exchange competition. It was sponsored by G-Research ( It’s a fairly efficient way of hiring the best employees). Unfortunately, we didn’t win (we won last year. Yay), but had we had more time we maybe would have. To give some context, the stock exchange market is a place where you have instruments (company shares) and their respective market price. The price is not an absolute thing but defined by the average value at which the shares are being exchanged (and somewhat related to how the company is doing in real-life, but during the competition we didn’t consider that). There was a central mechanism which kept a record of all the transactions and offers in the market, and each participant ( a company in real life) could propose to buy/sell instruments. Whenever there is a match (I sell at a lower price than you buy, or vice-versa), the transaction occurs at the price the first of us set. We also had a ‘Black Box’ which gave us relatively accurate predictions of future prices for the instruments. By the end, the participant who makes the most money wins. The entire code was done in C# (They, G-Research, scripted the structure of the program, and then we had to complete the rest with our own idea).

We started with a very simple idea: We take (A) the average of the last transactions, compare it to (B) our black box prediction and decide to buy if A < B or sell if B > A. This was to make sure everything was operational, and then we tried to make a quadratic approximation of the values of the past few seconds in order to predict in a long-ish term how the stock values will change, and then decide to buy/sell using that value (and completely bypass the black box given to us). Unfortunately when we implemented it we didn’t manage to make it run our algorithm in the given time, so we had to run the simpler idea (The maths for the quadratic approximation had all been worked out, though).

During the live test, all teams seemed to have a similar behavior (they all improved at the same time, had the same spikes and, in general, showed the same trends). Even though some correlation is expected (because increasing the value of the instruments increases the amount of money everyone has), usually there are differences in behaviors which we didn’t see, so I believe everyone used the same algorithm, which makes it kind of boring.

Still, I wonder how it would be if such a system was set up online where people had all the time in the world to implement their algorithm of choice (Which happens to be the actual real money high-frequency stock exchange market). No doubt that is a lot more fun than this mini version we did. I might pitch in on this some day.

Oxford University Elliptic Curves Research Project

2015-09-18T00:00:00+00:00

Computing the average rank of elliptic curves in ordered families

Supervisor: Dr Jennifer Balakrishnan

An elliptic curve $E$ is the set of points $(x, y)$ satisfying an equation of the form $y^2 = x^3 + ax + b$, together with a special point “at infinity”. When the coefficients $a$, $b$ are rational numbers, it is of interest to study the set of all rational points $(x, y)$ satisfying the equation. It turns out that the rational points on $E$ form a finitely generated abelian group, and an important invariant attached to $E$ is the rank of this group. During the project we explored the “average” rank of various elliptic curves in certain families by using Sage to build a database of millions of elliptic curves, then computing various important number theoretic invariants attached to these curves.

Farfetch Internship

2014-07-23T00:00:00+00:00

Applying Machine Learning to Fraud Detection

Supervisor: Cristina Cerqueira

When I first joined Farfetch I was told they were looking for someone to implement Machine Learning algorithms to help predict possible fraud attempts. These fraud attempts can consist of something as simple as using someone else’s credit card to make purchases to more convoluted methods which I might not be aware of. I was unaware (that is, until my first day there) that I was going to be the only one (at the time) working on it. Hence I dived into learning more about SQL and the Pandas module in Python (both of which I was working with for the first time) and into Machine Learning algorithms. I used Microsoft’s Azure Machine Learning Studio for a quick testing of different Machine Learning algorithms, and I repeated the same in R to double-check the accuracy of different algorithms. Both agreed that Random Forests was the most efficient method.

Having decided on Random Forests for the initial implementation in Python. During the next three weeks came the manual labor: building the queries to retrieve all the needed that in an efficient manner (where information was not only across different tables but also different databases, which slowed down the process) to transforming the raw information into more relevant data using tips from field experts at Farfetch, to checking that datatypes match,… and finally running the Machine Learning algorithm on the transformed data. After all this the python/sql scripts worked as expected. I quickly tried the same scripts with different algorithms to check (again) for efficiency, but similarly to the initial tests Random Forests performed best.

All of the above was done on static data. I still had to make it so that the algorithm could run live. The speed of the script wasn’t an issue, but I had yet to understand the way Farfetch pipelines so that I could integrate my work. Unfortunately my internship was coming to an end (and I had to go back to my studies in October). Hence I spent my 2nd last week organizing my code (It was a mess when I started, but by the end it was incredibly neat), and writing plans/suggestions/documentation for what exists/could be improved/needs to get done. In the last week a new intern, Nuno Monteiro, came along to continue this project (which by then got the informal nickname of project Sherlock). During that week my work was to put him up to date on the work that was done.

Nuno was doing his Master thesis at Farfetch, specifically about implementing Machine Learning in industry (titled A data mining based system for credit-card fraud detection in e-tail) , so he wrote his thesis alongside this internship (and I got to co-author the paper given my initial role in Project Sherlock).

Looking back I reckon I would’ve been more productive if I practiced more SQL/Python and less time on understanding the theory behind the Machine Learning algorithms used, but they were so interesting (As much as I like coding, there will always be a mathematician in me curious about how things work, and not just in making them work).

I’d also like to thank João Sousa (Pandas expert) and Carlos Gomes (SQL expert) for all the technical support they’ve given me during the internship, and everyone else who was there during breaks and lunch and gym sessions and dinners out. Couldn’t have had a better internship.

Delfos Project & Math Olympiads

2013-09-29T00:00:00+00:00

This was my last Olympiad. I first heard about Delfos when I was 14. I got to a Maths National Olympiad final and was told about this Maths Weekend school. Boring, I thought, so I didn’t join. 2 years later I happened to meet someone who went to Delfos and convinced me to try it (Thank you Catarina!). So I went. Expectations were low, but it was worth the shot (I already did maths in my spare time, but the idea of spending my weekends sitting in a classroom didn’t seem too appealing. I believe most of you will relate). How wrong was I. There were a few teachers who guided us a great deal (a special thanks to professor Kovacec who always went the extra mile for all of us), but the mutual learning was what amazed me. Yes, there were lectures, but we could just sit in the back doing something else or walk outside if, for any reason, we weren’t feeling productive (or if we just didn’t feel like it). But there I was, surrounded by people who were motivated to learn maths out of their own will, for fun. People who enjoyed the tricks and details observed in proving tricky problems, and were there for the challenge and the beauty of these proofs. We challenged each other with increasingly harder problems, and we improved ourselves.

This is the environment that I was lucky to have found. Together, we solved problems and became friends. Summers ran by as we spent weeks at a time in each others’ houses, swimming in the ocean, playing Playstation, camping in Gerês, playing Mafia till 3pm and solving IMO’s. We grew together. It’s funny how I thought Delfos would be this dull place where people stare at books and learn how to mechanically solve problems.

As wrong as I was, we did solve problems and even went abroad for some Math Olympiads. My first one abroad was in Brazil. Foolish me thought: Brasil is warm,I can just use flip-flops all the time. Wrong. I should have taken trainers. I hadn’t been there for a day but decided to play tennis (there were tennis courts nearby). Shoeless. Under the scorching sun. I got blisters. I couldn’t stand on my own two feet. For the rest of the week I had to use canadians. I will never make that mistake again, but at least I got to enjoy Salvador da Baia. The streets are surprisingly similar to Porto’s, due to our “calçada”, even more so than many Portuguese towns.

It so happened that my birthday was during that Olympiad. They got me a surprise cake and Pedro, from Angola, was put in charge of getting me to the cake. I was playing Catan at the time with my Portuguese friends (who were unaware of the surprise), so we finished the game. It only took an hour of 20ish people waiting. After finishing the game Pedro tried to convinced me to go to the swimming pool (the cake and the people were next to it), but it was dinner time and I was hungry, so I went for dinner with the rest of the Portuguese. Another hour went by and everyone was still waiting. Finally we went to the pool and there they were. I was really surprised (If I had any suspicion of such they wouldn’t have waited for 2 hours). The guide from Timor Leste convinced me to follow his tradition of breaking 17 raw eggs on my head (as I was turning 17), and after a quick rinse in the shower we all went to the pool. I will not forget this birthday.

I had my next Olympiad in Mozambique, but since I’ve been there before (to visit my brother, who worked in Maputo at the time), there wasn’t the novelty factor in it anymore. The last trip abroad was in Panama, in the ciudad de Panamá. During this trip we were escorted at all times by guides, and you can’t imagine how disappointed I was at crossing Panama’s Canal while not having had the chance to swim in neither of the Oceans bordering it (I managed to do so in both Brasil and Mozambique). It was an awesome week with Miguel (me), Miguel (Santos), Miguel (Moreira) and Luís (Duarte). However the ending was stressful for my parents. How so? Well, the flight took some 14 hours (with a schedule somewhere in Europe, I actually forgot where. It doesn’t matter, we never left the airport, so let’s say it was Frankfurt). We departed from Panama early in the morning and arrived in Portugal (Lisbon) at 9pm ish, after wich I had a 3h train trip to Porto. As such I emailed my parents the day before telling them I would arrive in Campanhã’s train station by midnight ish. I did not specify the day (which I assumed my parents knew). They assumed I would arrive on the day I sent the email. When they got to the train station I wasn’t there. They waited and waited. They went to the cops, who promptly told them there is a 24h mandatory wait period before initiating searches. My parents called my friends trying to figure out if they knew were I was. All the while I was sleeping. I woke up in the morning, didn’t check any form of communication cause I had no reason to, and I went to the airport. I got in, I started the flight. My parents still had no idea where I was, but by then they were hoping that I would arrive the next day. Something about the idea of me getting kidnapped in Panama wasn’t pleasant to them, so they hoped. I finally arrived at Frankfurt, with a call from Bea (a high-school friend of mine). She told me to call my parents since they were worried sick because of me. I did so, and my father was surprisingly calm as I spoke (probably relieved after all this). I boarded the last leg of my flight, had the 3h trip with another friend of mine from Porto and finally arrived home. Everything ended well. Except for my parent’s heart. That couldn’t have done them any good.

As a final note, I leave the list of all Olympiads I was lucky enough to have gone to, all in Mathematics unless otherwise stated. Happy times.

Date and Place	Olympiad	Medal
September 2013, Panama	Ibero-American¹	Bronze
August 2013, Mozambique	OMCPLP²	Gold
March 2013, Portugal	National	Silver
November 2012, Portugal	Paulista³	Silver
July 2012, Brasil	OMCPLP²	Silver
March 2012, Portugal	National	Gold
November 2010, Portugal	Paulista³	Bronze
May 2009, Portugal	National (Chemistry)	Silver
March 2009, Portugal	Regional (Chemistry)	Gold

_{¹ Participants: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, España, Guatemala, Honduras, México, Nicaragua, Panamá, Paraguay, Perú, Portugal, Puerto Rico, Uruguay y Venezuela.}

_{² Portuguese Speaking Countries´ Math Olympiad.}

_{³ Portugal/Brazil joint Math Olympiad.}

Miguel Torres Costa

Wordle

Easy to State, Hard to Solve

Problem

Solution

Approach

Recursions

Answer

Mathsy Shortcuts

Geometric mean is the (arithmetic) mean of magnitudes

KPIs - Lag vs Lead Measures

Strange Attractors

Intro

Tool

Can you replicate these patterns?

Sierpinski Triangle

Sierpinski Tetrahedron

Sudoku

Big Sudoku

Octogon

Cursor

Falcon

Perspective

Tic Tac Toe

Trapezoid

All horses have the same colour

All horses have the same colour pattern

MasterMind - Game

Goal

Asset Division

Goal

Technical Note

Steps

Integrating Factors

Intuition

General Solution

Example

Priority setting - should we train guide dogs for the blind?

Key ideas

Dimentions available to show information.

Possible information shared

Rough action steps

Shopping Cart

Pricing Information

First Sign of Civilization

Perfect Forecasting

Table of Contents

Assumptions for this quest

Event outcomes are not just true and false

We cannot observe the underlying probability of events, only the outcome

The value of better forecasts can be quantified

Existing companies in this area

Good Judgement Project

Foretold

Metaculus

Augur

Forecastapp aka Facebook

The Future

The Hardest Logic Puzzle Ever

Finding the solution

But we can’t distinguish the words for yes and no. Doesn’t that reduce the answers we are able to distinguish in half, since if we swap all of the da’s and ja’s we would get the same information?

Awesome, but what about the Random God?

What do you mean bits of information?

Alright, so how do we go from here to an answer?

Case A

Case B

Case C

Case D

So we’re done, hurray!

The answer

Online Mafia Rule book

Game Overview

Requirements

Setup

Characters

Practical notes

Mafia 2.0 (Ongoing)

Original Roles

Players (15 alive)

1st day message