A nice trick for renaming large numbers of files in Matlab:
Using Java code to rename files in Matlab
Thursday, February 21, 2013
Wednesday, February 20, 2013
Correcting for Familywise Error
When conducting a group of statistical comparisons, it is important to understand the concept of familywise error or the 'familywise error rate'. The concept of familywise error describes the probability of incorrectly rejecting the null hypothesis. Perhaps more informally, familywise error represents the odds of reporting a statistical difference when no such difference exists.
To understand why familywise error is important we must revert back to concepts learned in a basic introductory statistics class. First, we generally compare t-values (or other test statistics) to a known distribution of scores (t-scores in this case). Like any distribution, some values near the center of the distribution are more likely to occur than others. By understanding the odds that a given t-score will occur, we are then able to designate values that define regions of the distribution that are unlikely to occur as "critical values". That is, values above or below these critical values only occur with a certain limited probability.
The exact probability we choose to define the critical values is arbitrary and often driven by tradition. As mentioned previously, we want to select values that are unlikely to occur in order to reliably report statistical differences. On the other hand, selecting values that are too stringent might cause us to report that no difference between our samples exists when in fact a difference does in fact exist. In psychology, a alpha of 0.05 is generally designated and denotes that falsely rejecting the null hypothesis should only occur 5% of the time. This value has been shown to provide a nice balance between selectivity and reliability.
Having said this, it is important to realize that the probability of finding a test statistic that is outside of the critical values for a single comparison is 1 in 20 (5%). However, we also know from probability that the more chances one has, the more likely an event becomes. For example, if a player's chances of winning the lottery with one ticket are 1 in 14 million, that player can increase his chances to 1 in 14,000. As you might imagine, this presents a problem when running multiple statistical comparisons. The more statistical tests we run, the more likely it is that we will incorrectly identify a statistical difference (i.e. incorrectly reject the null hypothesis).
There are many ways to correct for this problem. Each different correction has its pros and cons and each correction may be more or less stringent than the next. Thus, one must consider how conservative of a correction a given situation calls for, if a desired correction will be overly liberal or conservative given the number of comparisons being made, and perhaps if the number of comparisons being made even warrants a correction at all.
One common correction that you might use is a Sidak correction. This is a simple correction to calculate using the following formula:
1−(1−α)^(1/n)
where, α represents your chosen alpha level and 'n' represents the number of comparisons being made. This formula returns a corrected p-value that should be utilized in in lieu of alpha for each of your multiple comparisons when deciding whether or not to reject the null hypothesis. For example, if one were to make five separate statistical comparisons, the Sidak correction would dictate that only p-values of < 0.01 should be considered instead of the typical p< 0.05.
Try utilizing the formula yourself to calculate a corrected p-value and see how increasing the number of comparisons necessitates a greater correction.
D.J. Barker
To understand why familywise error is important we must revert back to concepts learned in a basic introductory statistics class. First, we generally compare t-values (or other test statistics) to a known distribution of scores (t-scores in this case). Like any distribution, some values near the center of the distribution are more likely to occur than others. By understanding the odds that a given t-score will occur, we are then able to designate values that define regions of the distribution that are unlikely to occur as "critical values". That is, values above or below these critical values only occur with a certain limited probability.
The exact probability we choose to define the critical values is arbitrary and often driven by tradition. As mentioned previously, we want to select values that are unlikely to occur in order to reliably report statistical differences. On the other hand, selecting values that are too stringent might cause us to report that no difference between our samples exists when in fact a difference does in fact exist. In psychology, a alpha of 0.05 is generally designated and denotes that falsely rejecting the null hypothesis should only occur 5% of the time. This value has been shown to provide a nice balance between selectivity and reliability.
Having said this, it is important to realize that the probability of finding a test statistic that is outside of the critical values for a single comparison is 1 in 20 (5%). However, we also know from probability that the more chances one has, the more likely an event becomes. For example, if a player's chances of winning the lottery with one ticket are 1 in 14 million, that player can increase his chances to 1 in 14,000. As you might imagine, this presents a problem when running multiple statistical comparisons. The more statistical tests we run, the more likely it is that we will incorrectly identify a statistical difference (i.e. incorrectly reject the null hypothesis).
There are many ways to correct for this problem. Each different correction has its pros and cons and each correction may be more or less stringent than the next. Thus, one must consider how conservative of a correction a given situation calls for, if a desired correction will be overly liberal or conservative given the number of comparisons being made, and perhaps if the number of comparisons being made even warrants a correction at all.
One common correction that you might use is a Sidak correction. This is a simple correction to calculate using the following formula:
1−(1−α)^(1/n)
where, α represents your chosen alpha level and 'n' represents the number of comparisons being made. This formula returns a corrected p-value that should be utilized in in lieu of alpha for each of your multiple comparisons when deciding whether or not to reject the null hypothesis. For example, if one were to make five separate statistical comparisons, the Sidak correction would dictate that only p-values of < 0.01 should be considered instead of the typical p< 0.05.
Try utilizing the formula yourself to calculate a corrected p-value and see how increasing the number of comparisons necessitates a greater correction.
D.J. Barker
Friday, February 15, 2013
Gas Chromatography explained
After our discussion today, I went looking for some tools to help illustrate the process for gas chromatography. Below you will find a schematic of the GC equipment and a great video by thermo scientific demonstrating how a column works.
Wednesday, February 13, 2013
Techniques in Neuroscience
For those interested in a crash course in neuroscience techniques, I would highly recommend the book entitled Guide to Research Techniques in Neuroscience. I recently purchased this book and have been examining it most of the evening. Techniques covered in the book include, but are not limited to:
-Imaging
-Animal Behavior
-Stereotaxic Surgery
-Electrophysiology
-Microscopy
-Genetics
This book will not necessarily give you the necessary skills to perform the techniques listed, but will give you a well-rounded overview of the techniques that might allow you to more effectively read, critique, and effectively discuss studies with techniques that are outside your purview. More importantly, this book provides a good middle ground between typical texts that are either oversimplified, overly-technical, or disconnected from the "hands on" side of neuroscience.
D.J. Barker
-Imaging
-Animal Behavior
-Stereotaxic Surgery
-Electrophysiology
-Microscopy
-Genetics
This book will not necessarily give you the necessary skills to perform the techniques listed, but will give you a well-rounded overview of the techniques that might allow you to more effectively read, critique, and effectively discuss studies with techniques that are outside your purview. More importantly, this book provides a good middle ground between typical texts that are either oversimplified, overly-technical, or disconnected from the "hands on" side of neuroscience.
D.J. Barker
Saturday, February 2, 2013
How to Shape Behavior
In behavioral experiments, training animals to reliably perform the task is crucial to the success of the task. Still, finding ways to produce behaviors--no matter how simple--can prove challenging for any research scientist. Not to worry. Teaching animals to perform a desired behavior can be accomplished through a process called "shaping". More specifically, shaping is a method for conditioning a behavior and was first introduced by B.F. Skinner. The procedure involved gradually changing behavior over time until a certain target behavior is reached.
One example of shaping might involve teaching a child to write his or her name. This might first involve praising the child only for correctly forming the first letter. Eventually, in order to reach the goal of having he child write his/her full name, it becomes necessary to raise the bar until the child has written the second, third, and eventually all of the letters in the name. Importantly, successive approximations also allow for failure by only raising the bar a little at a time. Thus, the second rule of shaping is that learning involves making mistakes. Certainly, it is just as important for an individual to learn what NOT to do as it for them to learn the desired behavior. To continue with the above mentioned example, it might be important for a child confusing the letters "M" and "N" to correctly learn to discriminate these letters in order to correctly spell his/her name.
This brings us to our first "rule" of shaping: Learning is a gradual process. This means that shaping a new behavior involves patience. One technique used to "patiently" shape behavior involves "successive approximations" of the target behavior. The use of successive approximations involves reinforcing behaviors that come close to the target response without necessitating that a subject actually emit the target behavior.
One example of shaping might involve teaching a child to write his or her name. This might first involve praising the child only for correctly forming the first letter. Eventually, in order to reach the goal of having he child write his/her full name, it becomes necessary to raise the bar until the child has written the second, third, and eventually all of the letters in the name. Importantly, successive approximations also allow for failure by only raising the bar a little at a time. Thus, the second rule of shaping is that learning involves making mistakes. Certainly, it is just as important for an individual to learn what NOT to do as it for them to learn the desired behavior. To continue with the above mentioned example, it might be important for a child confusing the letters "M" and "N" to correctly learn to discriminate these letters in order to correctly spell his/her name.
A third rule of shaping is that reinforcement must be given in a consistent, steadfast manner. When reinforcing approximations of a target behavior, it is important never to lower the bar once a behavior that is more proximal to the goal behavior has been learned. For example, it would not be effective to reward a child who had learned to spell half of their name for only inscribing the first letter. On the other hand, one must also be careful not to raise the bar too quickly. It is important to consistently reinforce each "step" in the approximation process for long enough to ensure that the subject has mastered the behavior before moving on.
The last major rule for shaping is that an experimenter must learn to observe behavior. To most rapidly shape an animal, an experimenter must exploit innate behaviors exhibited by that animal. Realistically, an experimenter should take a naturally occurring behavior that best approximates the target behavior and begin to reinforce that behavior as an initial approximation. To do this, an experimenter MUST observe and understand an individual's behavior. The key word here is "individual", as no two animals will be shaped in the same manner. Thus, to shape a specific individual, one needs to have a well-developed understanding of the behavioral repertoire that an individual possesses.
Shaping a lever response in an highly mobile rat might involve reinforcing moments when the rat's innate ambulations place the animal in close proximity to the lever. Once the animal has learned that the quadrant proximal to the response manipulandum is special, it would be easy to begin raising the bar and better approximating a lever response. In contrast, a scared animal that freezes in the corner of the chamber could not be shaped using this same method (at least not easily). As you might imagine, shaping this animal would necessitate a very different strategy. One such strategy might be to spread pieces of food around the chamber so that our scared rat begins moving around to eat the food and better approximating our more mobile rat.
Lastly, it is important to realize is that shaping takes practice. Practice allows you to observe animals behaviors, develop an understanding of individual differences among animals, and to develop a plan in your head for how to carry out your shaping procedure. Each subject is different, and each task we might ask a subject to complete comes with new challenges. Still, despite all of these differences, the essence of shaping never really changes. Happy shaping!
David Barker
Subscribe to:
Posts (Atom)