Statistics - Sampling Methods and Estimation

In statistics we have to use samples because it's normally near on impossible to get data for the entire population. As long as the sampling is done well, the results will usually be good enough. Logic would tell you that the larger the sample, the better.. and this is true. There are two concepts we need to understand here, those are random sampling and sampling distribution.

• Random sampling - The goal of this is representativeness, we aim to get an equal probability of selection to every member of the population. There are a few methods:
• Simple random sampling - A sample so that every item or person in a population has the same chance of being included.
• Systematic random sampling - Items or individuals are arranged in some sort of order. A random starting point is selected and then every nth member is selected. Alphabetic order for example. 
• Stratified random sampling - A population is divided into sub groups (strata) and a sample is selected from each strata.
• Cluster sampling - A population is divided up into primary units and then samples are selected from the primary units.
• Non-probability sampling - Inclusion in the sample is based on the judgement of the person selecting the sample. (Eeek!)

• Sampling Distribution - This is the theoretical distribution of a statistic for all possible samples of a certain sample size, N. It's a device to link the samples characteristics to the population.
• If repeated sample sizes of size N are drawn from a normal population with a mean of mew and a standard deviation, σ, then the sampling distribution of sample means will be normal with a mean of mew and a standard deviation of σ / SqrRoot(N).
• The 'Central Limit Theorem' states that if repeated samples of size N are drawn from a population, as N becomes large the sampling distribution or sample means will approach normality.
• Or, in easier terms: Large samples are more reliable!

The more basic method of estimation is confidence intervals. From a sample we don't know the population mean, but we would like to estimate this with maximum efficiency. To do this we use a range, and say how certain we are that this range includes the population mean. We give a confidence interval in the form of a percentage, for example we could say that at a 99% confidence interval, between 33% and 39% of adults will vote for Labour in the next election (Made up!). A bigger confidence interval is more likely to contain the true population mean.

The next post will go further into the concept of confidence intervals and we will introduce such things as error margins. Stay tuned, thanks guys!

Sam. 

Statistics - The Normal Probability Distribution

This post will bring in an application of standard deviation. It can help give us units to measure distances between points in a data set as well as to measure the distance from the mean.

Chebyshev had a theorem. He said that for any set of observations, the minimum proportion of values that lie withing k standard deviations of the mean is 1- (1/k^2), as long as k is greater than 1. If k is 3, 89% of the observations lie withing the region and if k is 4, 94% of observations lie within the region.

For a normal probability distribution we need to use a normal curve, or a bell curve. It has a single peak in the centre of the distribution. This centre point is where the mean equals the median equals the mode. We can now introduce a new concept of z-values. A z-value is the distance between a selected value (Xi) and the population mean, divided by the population standard deviation. Another note on the normal curve is that is has a Kurtosis of 0. A higher Kurtosis means it's peak is higher and more pointy, a lower Kurtosis means it's a flatter shape.

Back to the z scores. They link together the theoretical normal distribution to the observed observations. It tells us how many standard deviations away from the mean an observation lies. So, we need to calculate the z-score. When calculating the z score it is essentially converting your data into a distribution with a mean of 0 and a normal curve shape. The formula is as follows:

Once the score has been calculated, you refer to a z score table. On this table, the first decimal goes down the side and the second decimal goes along the top. So, if using the formula above you were given a z score of 1.24, then you'd look for 1.2 down the side and 0.04 along the top. The score at which these match is your z score. That score is 0.3925. What to do with that score becomes more understandable with an example and some context. 

At a party, lemonade is distributed among the party-goers with a mean of 20cl and a standard deviation of 5cl. What is the likelihood that a person selected at random will get between 17cl and 23cl of lemonade? Right, so we plug in the values to start. z will equal (17 - 20) / 5 = -0.6. It will also equal (23-20) / 5 = 0.6. Now, from here we look for 0.6 down the side of the z score table and 0.00 along the top. We will be given a value of 0.2257. This score caters for one of the results, but as they are both the same size we can double it and get 0.4514. And that's the answer. 45.14% of people get between 17 and 23cl of lemonade, so the likelihood of any given person getting between that amount is 45.14%.

Another use for z scores isn't just finding the amount included, it can be used to find excluded regions too. That may sound quite complex, but I'll show you a picture to visualise it.

We need some context once again to work this out. Let's say that a teacher has said to achieve an A* on a test, students must get in the top 10% of the scores. The mean score for the test was 75 and there was a standard deviation of 5. We can now work out what score is needed to get a A*. The whole region to the right of the mean makes up 50%, we know we want to exclude 10%, so we need to find a z-score that marks 40% - 0.4. On the z-score table 0.4 doesn't appear (remember this time we know the size of the region, so we are looking at the values in the table and looking for a corresponding z score), 0.3997 is the closest so we'll go with that. That gives a z-score if 1.28. Now we need to refer back to the z score formula. We know z, we know s and we know the mean... we are trying to work out Xi. So, we plug in the numbers we have an rearrange to find Xi. 1.28 = (Xi - 75) / 5. Xi - 75 = 6.4. Xi = 81.4. There we have it, the answer. To make it more realistic this score could be rounded to 81 or 82, but one of these scores is needed to achieve the top 10% of the class and therefore get an A*. Simples!

That's it from me, z-scores are a fairly complex topic so feel free to ask any questions if i haven't been entirely clear in the explanation. Good luck!

Sam.

Statistics - Measures of Dispersion

The first measure of dispersion of data we'll mention is the range. The range is the difference between the highest value and the lowest value in a set of data. Only these two values are used in the calculation and it is very easy to compute. It has a slight issue, however, that extreme values do influence the result. 

A step on from this range is the interquartile range. This is the difference between the first quartile and the third quartile - giving us the middle 50% of observations. To work out the first quartile, we take N (number of observations) and divide it by 4. This will give us the number of the observation at which the first quartile point is. To work out the third quartile we take N and divide it by 4 and then multiply the result by 3. This gives us the number of the observation at which the third quartlile mark is. From there, we just subtract the value of the first quartile figure away from the third quartile figure. 

The quartiles are then displayed on a box plot diagram. A box plot will look generally as follows:

Mean deviation is another measure of dispersion. This measures the mean of the absolute values of the deviations from the mean. Similar to standard deviation, but not quite. The formula is as follows:

• Mean Deviation = (Σ|Xi - x̄|) / n
• Xi = each observation
• x̄ = the sample mean.
• n = number of observations.

We take the absolute values here for a very specific reason. It stops the negative and positive values from cancelling each other out, which would give us a mean deviation of close to 0 - very unhelpful! Dispersion is very important, key statistical methods such as regression rely heavily on measures of dispersion. 

The population variance and sample variance are two more concepts I'm going to introduce now. The population variance measures the arithmetic mean of the squared deviations from the population mean. The sample variance essentially does the same, but for a sample. The formula for both are as follows:

• Population variance = (Σ|X - μ|)^2 / N
• Sample variance = (Σ|Xi - x̄|)^2 / (n - 1)

These variances can be easily turned into the standard deviations, a very important concept for statisticians. To do this, we just square root the result. We denote the standard deviation of a population and a sample differently. A population is given with this symbol: σ and a sample is given with the letter s. The standard deviation principle will come in key in the next few posts when we begin to introduce confidence intervals, so learn it!

Thanks for reading, have a good day.

Sam.

Statistics - Measures of Central Tendancy

By the end of this post I hope that you'll be able to characterise a data-set with one piece of information  and more importantly briefly describe complex data in simple terms. 

We'll start with the mean. The mean is computed by adding up all the values and dividing by the number of observations there are (N or n). Two symbols appear for means, these are x̄ (sample) and μ (population). If you take each score in a distribution and subtract the mean from it, and add all these differences the sum will always be 0. The mean does have some disadvantages, such as extreme scores pulling the mean one way or another. This issue doesn't occur with the median. To work out the population mean, or μ, all data in that population must be added up and divided by the population. For the sample mean, or x̄, all sample data must be added together and divided by the number of observations in the sample. 

A different type of mean to the arithmetic mean above is a weighted mean. This allows us to create accurate calculations even when all the information isn't known. It's fairly straight forward, like above. The formula for weighted means is as follows: x̄ = (w1X1 + w2X2 + ... + wnXn) / (w1 + w2 + ... + wn). 'w' here denotes the weight given to the value 'X', the higher the weight the more influence it has on the mean. If the weights are all 1 you essentially have the same formula for as the arithmetic mean in the previous paragraph. The problem with this is that sometimes the weights aren't known and outliers are very common in economics. 

Next, we move on to the median. This, as many already know, is the middle score when the observations are arranged in order. If there is an even number of scores, it's the mean of the middle two values. There is a unique median for each set of data. It isn't affected by extreme values and is therefore a very good measure of central tendency. 

The logical step now is to introduce the mode. The mode is the most frequently appearing value in a data-set. It is of limited use because it doesn't give any weighting to unique values. Other problems arise, such as some data having no mode and some having more than one. It really doesn't give a good measure of central tendency for a set of data. 

Skewness! A few graphs will crop up in this section. Skewness tells us in which direction the data swings, it is normally represented graphically. We start off with the bell curve. This is when the mode = median = mean, we have no skew and the data fits nicely into this 'bell' shape shown below. The distribution is symmetrical. 

A positively skewed distribution occurs when the mode < median < mean. The data is all bunched to the left of the mean and then it falters out. It looks like below.

Finally, the negatively skewed distribution. It's the stark opposite of above, the mean < median < mode. Here it is. 

You might have worked out by now, by skewness measures the lack of symmetry of the distribution. This lack of symmetry can be given a numerical value ranging from -3.00 to 3.00. A value of 0 indicates a symmetric distribution. It is worked out as follows: sk = 3(x̄ - Median) / s. 's' stands for standard deviation which will be coming up in the next blog post, so stick around for that. 

That's all for this post. I hope now you're able to describe your data set fairly easily, tell us the middle value, the average in which direction it's skewed and so on and so forth. It's a good tool to have, yet still very basic. We'll be stepping up another gear next post as range and deviations are introduced. Keep checking back, thank you guys!

Sam. 

Statistics - Introduction

*Disclaimer: I have a statistics test coming up soon, so expect the next few blog posts to be all statistics related. I'll be getting back to normal economics after the 8th December. Statistics, however, is useful to people outside the discipline of economics as well as econometrics studies.*

I'll start at the very basics of statistics. This post will contain a few definitions and a few formulae, nothing too taxing as we're just setting the scene, so to speak. In statistics, we have two types of variable; a qualitative variable and a quantitative variable. The features of these are as follows:

• Qualitative: Non numeric. For example: gender, religion.
• Quantitative: Numeric. For example: bank balance, age. Can be either discrete of continuous.
• Discrete - Can only have particular numbers. Example: Family members (Only whole numbers)
• Continuous - Anything else. Example: Weight, height.

Data can define on different levels as well. We can have nominal, ordinal, ratio or interval data. Again, these all have different characteristics:

• Nominal Data - Categorised data that cannot be arranged into an order. Eye colour, for example. This data can be either mutually exclusive and/or exhaustive. It's usually qualitative.
• Mutually exclusive - Can only be included in one category. Only one eye colour for example.
• Exhaustive - The data must appear in at least one category.
• Gender is mutually exclusive and exhaustive.
• Ordinal Data - Data that can be arranged into some sort of order. Individuals can be compared with one another here because of rankings, however the distances between each piece of data has no meaning.
• Interval Data - Can tell differences between the distance of data, yet we still aren't able to say that 100 is twice 50.
• Ratio Data - Has a 0 point, e.g number of family members. Now we can say that a family of 4 members is twice as big as a family of 2.

I'll now move on to looking at frequency distributions and how we can present these graphically. A frequency distribution is a grouping of data into mutually exclusive categories showing how many observations are in each class. A class is a subset of the whole range of the data, each class will be the same size and all the classes together will make up equal to or more than the range of data.

The class midpoint is the average of the upper and lower limits of the class. The class frequency is how many observations fall into that class. The class interval is the difference between the upper and lower limit of the class. 

There is a rule for how to divide your data into classes. It's named the 2 to the k rule. In this example we refer to 'N', this is the number of observations. The rule is you choose k so that 2 to the power of k > N. This value of k will be the number of classes. How to choose the size of these classes is fairly straight forward as well. Minus the upper score by the lower score and then divide this value by the k we just got from the last formula. A decimal value may well be given, in that case round UP to the nearest sensible number. The data can now be displayed clearly in a frequency distribution diagram/table.

Relative frequency distribution is essentially the same as above, except adding one more column in the table. This new column shows the percentage of observations in that class. Divide the amount of observations in that class by the total amount of observations and multiply by 100 to get the percentage. Simplez.

The data can be displayed graphically in many forms, none of them being 'wrong' per say. Histograms, frequency polygons and cumulative frequency distributions are examples of these graphical representations. These are all for examining data and trends in that data.

You may ask the question about line graphs here, when do we use these? In general, line graphs are used when time is involved. They can show the change over a period of time. Bar charts are for showing different categories that have no clear link, or to measure frequencies. Pie charts are simply for showing proportions. Scatter graphs show us co-variation of two variables, but NOT time. Any over diagram are subjective and not advised. 

Introduction to statistics complete! I hope that makes enough sense to you all, comment your problems if not and I'll get back to you ASAP. Thanks for reading, good luck!

Sam.