Învăţare automată

Elementary probability theory

1. A fair coin is tossed 3 times. What is the chance of 2 heads?
2. Are X and Y as defined in the following table independently distributed?

   X	Y	P(X, Y)
   0	0	0.32
   0	1	0.08
   1	0	0.48
   1	1	0.12

3. Prove that:
   • P(A₁ n ... n A_n) = P(A₁)P(A₂|A₁)P(A₃|A₁ n A₂) ... P(A_n| n_i=1^n-1 A_i)
4. Suppose one is interested in a rare syntactic construction, perhaps parasitic gaps, which occurs on average once in 100 000 sentences. Joe Linguist has developed a complicated pattern matcher that attempts to identify sentences with parasitic gaps. It's pretty good, but it's not perfect: if a sentence has a parasitic gap, it will say so with probability 0.95, if it doesn't, it will wrongly say it does with probability 0.005. Suppose the test says that a sentence contains a parasitic gap. What is the probability that this is true?

Essential information theory

1. Compute the entropy of the set of possible outcomes (denoted X) for rolling an 8-sided die.
2. Simplified Polynesian appears to be just a random sequence of letters, with the letter frequencies as shown:

   p	t	k	a	i	u
   1/8	1/4	1/8	1/4	1/8	1/8

   Compute the entropy of this language.
3. Write a program which receives a text at the input and computes:
   • the relative frequency of each letter
   • the entropy

Sursa: Manning & Schutze: Foundations of Statistical Natural Language Processing