By the Numbers

can we make this a free article so we can share it with the non-believers?

As we push to know more about the affects of nutrition, this is a great step. However one of the things I continually see missing from these types of studies/challenges is what affect 8 weeks of Crossfit has on the same people with their normal nutrition. Looking at the numbers in this study, this is not outside the realm of improvement just for a normal diet and someone doing Crossfit.

I would encourage anyone doing a challenge or study in the future to plan it out far enough to get the clients to participate in a paleo free/diet neutral cycle of equal duration before going paleo. This should set a baseline to try and find what kind of affects Crossfit has on the participants. Then we can truly measure the impact of paleo on clients with a higher degree of certainty.

I would agree a great step in the right direction, but I think still a very narrowed approach. However, we all know Crossfit stands behind zone and that is their choice of nutrition. (I hate the word diet).
Have to agree with Michael also that there is no placebo population.

I would like to see a study with:
No nutrition change w/ Crossfit.
Paleo w/ Crossfit
Zone w/ Crossfit
and
Paleo Zone with Crossfit.

then compare results.

I have done all variations in my own journey this past 15 months and have seen results with all, but my best results of health, metcon times, and strength gains came with straight paleo no restrictions to portion.
I just overall felt better. I didn’t lean out as fast as zone, but performance increased for myself and I still leaned out.
Full disclosure though I have lost over a hundred pounds with crossfit and nutrition changes. But I follow a paleo plan and eat when I feel hungry but I do have potion control in mind, as a form fat guy I am still battling the demons I see in a plate of food.

I'll echo Michael in stating that this is a very good start. I agree that the most practically feasible way to improve upon it would be to include identical measurements from a control group who does not follow a paleo/zone diet. They would of course also need to submit food diaries.

To provide a certain level of control over training variables between centers in a multi-center study one could agree to conduct the study under a specific two-month period using mainsite WODs.

Doing the study in this fashion would build upon the work that Amy and Tim have started in a very logical manner. Having a control group would help answer the question of how much progress is due to diet vs. training, and the larger sample size would increase the statistical significance.

Building further upon the study could be done over the same two-month period the following year, perhaps involving the additional groups that Khristian suggested along with an additional increase in sample size. If it were deemed necessary one could even use the exact same set of workouts in the same order that were presented in the previous test to further minimize the variables and allow the test results for both years to be additive rather than simply comparative.

90% Paleo isn't paleo....

Great start! Agree with the comments above and encourage you to conduct another study, perhaps with multiple researchers at multiple boxes with a larger population including controls. Good clinical research in nutrition is lacking and you could add a lot to the current literature.

Please also conduct follow up with your participants, perhaps at 6 mos and a year to see what happened to the lifestyle changes, did they stick, did they maintain their gains, did the continue to improve. I'm sure you're ahead of this already. Keep up the great work.

Observation and measurement presented with a road map for repetition...fantastic. What a great starting point.

Hi, thank you for your feedback, and I wanted to respond to a couple of the comments right away.

First, let me get the science out of the way. I completely agree that a controlled study (with a non-Paleo control group) would allow us to make stronger conclusions about causality. Unfortunately, we could only work with the available population, and we couldn't garner enough (or any!) people willing to participate in a study who weren't also interested in trying Paleo. At first my biggest wish was for scientific rigor, but as I saw that the control group wasn't going to pan out, the goals shifted a bit to: 1) getting people to try Paleo in a non-competitive setting and 2) setting up and documenting a template for other affiliates to follow. While this data is only suggestive because of the compromises we had to make to experimental design, we hope that it accomplished these other two goals.

On a related note, suggestions for follow-up work are great, but at our relatively small affiliate, where at least half of the members are already eating Paleo, we simply don't have a large enough population to draw from. The idea of a study spanning several affiliates is a great one, and I would be thrilled to consult on or administrate such a study. If your affiliate is interested in participating in something like that, please contact me and I'll see what we can get going.

Second, I recognize the concern that 90% Paleo is not at all the same as 100% Paleo. In human research, issues of compliance are always difficult. We gave people the 90% guideline because it was feasible. We did tell them that ideally they should do 100% Paleo!

Third - and this is an anticipated question! - I want to head off concerns about the Zone portion of the study. I am not in any way wedded to Zone macronutrient ratios and stray quite far from them myself. We used Zone ratios as a starting point for people and adjusted from there. It was just a way to provide some more structure for folks who weren't quite sure how much of what to eat. Some adhered very closely to these ratios, and others adjusted to ratios that worked better for them.

Finally, I've had several requests to borrow materials or templates that we used. I will get to work on making these materials available as Google documents, so that anyone can access them. If you choose to run a similar study at your affiliate, I am more than happy to consult and give you some guidance on how to do the data analysis. Don't hesitate to contact me with questions.

I'm sure there will be more great questions and comments, and I will be checking back to answer them. I'm very excited to start engaging in this discussion with the broader CrossFit community.

I'd like to echo the comments about control groups and assigning cause and effect in these types of studies. Although this study is better than none at all, it can't assign cause and effect to the results because there are too many uncontrolled variables. For example, are the results due to the commitment to 3 workouts a week, the food selection associated with the paleo/zone diet, the simple calorie restriction of the diet (i.e is it type of calories or number of calories that's important), or a combination of these? Would you see the same results with a group of people that committed to the paleo diet and an exercise program other than crossfit? Or for crossfit and a different calorie restricted diet?

I think it's also important look at desired outcomes. I would argue that the best diet for someone looking to loose weight is very different than for someone looking for optimum athletic performance. I don't think it is a coincidence that the top finishers at the 2010 games don't follow a paleo or zone diet - Holmgren, Froning Jr, Spealer, Clever, MacKenzie Voboril (zoneish but not calorie restricted)... not sure about Thorisdottir.

Amy, can you tell us what the two "identified outliers" were? With sample sizes in the 9 to 13 range, the two outliers might provide interesting information. Were there other factors identified that might explain these two data points? Also, did any of the participants that dropped off the diet portion of the study continue with the crossfit protocol and complete the post study measures? That might be one way to start looking at the crossfit vs diet question.

Echoing previous statements, this is a great start though you would need a control group to isolate Paleo as the independent variable and it would be helpful to quantify what "90% paleo" means (e.g. are they eating gluten?)

However, the blood numbers are interesting from a practical standpoint of being able to say "if you do CrossFit and eat Paleo, your blood numbers will improve," which is a common concern folks have about the amount of fat and saturated fat they are eating on the Paleo diet.

Definitely a great start! I would love to see a Design of Experiments (DOE) approach, this would help take into account blocking effects, interactions and if done properly could reduce the sample size. http://en.wikipedia.org/wiki/Design_of_experiments

This is good for anyone who doesn't know their body and needs a start to figure out what they should do. But saying it works for everyone is a far stretch. I continue to make gains with my non-Paleo, whole food diet. Everyone is different and diet logic must be applied to this as well.

control group?

Please see my earlier comment.

Weight loss... so now Crossfit is a weight loss program? I thought it was a "strength and conditioning program". Oh and how exactly would strength increase if you lost weight? A few months ago I lost 2 kg and my strength decreased tremendously as a result. Anyway, at least it seems like the paleo diet works for some people.

Lynne (and others who have this concern), I understand the issue with control groups, and therefore I was careful not to make any claims about causality in this paper. I simply wanted to present the approach we used so that others could use it as a template. If people want to make cause and effect conclusions, I would highly recommend that they try to get a control group (which in our circumstance turned out to be unfeasible, but that may not be the case at all affiliates). A lot of the confounds could be addressed by including various factors as covariates in regression analyses after the fact. If anyone is interested in running a similar study and wants this level of vigor in design and analysis, I am happy to consult.

Thanks for this detailed study. As you say in the article, "More research is needed to confirm the results of the study, and every CrossFit box is a lab." I see it more as providing info to help us understand nutrition and fitness better, than the final word on the matter.

I personally lost 45 lbs and improved my fitness last year by following the Zone plan. That was before CrossFit or Paleo. I joined CrossFit BWI early this year and participated in a 60 day Paleo challenge. I've been a type 2 diabetic for 18 years. My blood work showed significant improvement while on the Zone plan. However, 30 days into the Paleo challenge I was able to stop taking my diabetic medication. I saw my endocrinologist after three months on Paleo with no meds and my blood test results were better than they were on Zone with meds. She affirmed that I don't need to take the medication anymore, and just continue with my current fitness and nutrition lifestyle. That impact was even more significant than the performance gains realized during the challenge.

So, for me, this study confirms what I've experienced personally when combining Paleo eating with CrossFit exercise.

"90% Paleo isn't paleo...." - Says who?

Crossfit is a complete fitness methodolgy that is used by each individual to accomplish the goal(s) they wish to accomplish. If someone needs to lose weight, Crossfit is going to help (while increasing the person's strength and conditioning). If someone is a high level athlete in another sport, Crossfit can be taylored to improve the said person's strength and conditioning which will generally translate into better results on the field, court, slope or whatever. Size and strength aren't always directly proportional, so yes, one can lose weight and gain strength. In fact, I would say this is a common theme for most Crossfitters. A competetive powerlifter who is approaching their maximum strength capacity will lose some strength if they lose weight, but the majority of us aren't in that boat. A person who is out of shape, never lifted weights beore, and is 40 lbs overweight will most likely lose weight and gain strength. It all depends on each person's level of fitness and the goal(s) they wish to accomplish.

your comment seems unnecessarily purist about weight loss. Just because Crossfit methodology is based around performance as the primary goal, doesn't mean that people don't and shouldn't want to lost bodyfat and look leaner/more muscular.

The vast majority of people who walk into a crossfit box are doing so because they have a range of health and fitness related goals and body composition & improved appearance is almost always in the top 3. Given very few crossfitters can expect to excel in crossfit competition (even for the ones who actually want to), looking good on the beach, fitting into their skinny jeans or feeling better about themselves when they look in the mirror are achievements they can feel very proud of.

Sorry to hear that your strength decreased 'tremendously' as a result of a 2kg loss of weight. It is normal that some lifts will fall slightly as a result of lower bodyweight (eg back squat), but perhaps if the drops were more marked, it was a result of some other factor too?

Second the question about outliers. The criteria for exclusion did not seem to be documented in the analysis. Were they too fat? Ate too much sugar? What's the poop?

Observational studies where double blinding is not feasible are always difficult to interpret. The results substantiate the notion that CF + paleo led to body composition improvements in the participants, who possessed fairly typical demographics. Separating out whether a paleo diet had anything to do with this is another matter.

I didn't see any analysis of whether there was a relationship between the degree of compliance and the measures of improvement recorded. That might be interesting.

phenomenal job. I am holding a "Fall Leaning" challenge for my patients in my Chiropractic office, and this will be great ammo to help motivate them. Very well done and very well written. Thanks so much!!!

It is common for work like this to focus, at least partly, on blood markers of wellness. Commonly, these include the various cholesterol "standards" and triglycerides. These goals are well-accepted within the medical community. Of course, there is a multi-billion dollar pharmaceutical industry based around changing these numbers with drugs.

I would assume that most readers would accept the cholesterol targets of this and similar research without hesitation. We have been fed this doctrine for decades. For those readers curious about an alternate point of view, I encourage you to check out Anthony Colpo's "The Great Cholesterol Con". Most will find it a bit mind blowing. If you haven't the time or desire for this somewhat lengthy and detailed read, you can check out my take on it at http://getstrongergolonger.squarespace.com/journal/2010/8/9/feeding-the-machine-the-cholesterol-myth.html

This is still a couple of thousand words but will introduce those curious about the concept. Pretty crazy, really! This takes nothing away from the health benefits of the Paleo diet and training programs like CrossFit but simply suggests that we may be better off looking at other measures when assessing their value.

Get curious, question authority!

http://www.crossfit.com/cf-info/start-diet.html

"What Should I Eat?
In plain language, base your diet on garden vegetables, especially greens, lean meats, nuts and seeds, little starch, and no sugar. That's about as simple as we can get. Many have observed that keeping your grocery cart to the perimeter of the grocery store while avoiding the aisles is a great way to protect your health. Food is perishable. The stuff with long shelf life is all suspect. If you follow these simple guidelines you will benefit from nearly all that can be achieved through nutrition.

The Caveman or Paleolithic Model for Nutrition
Modern diets are ill suited for our genetic composition. Evolution has not kept pace with advances in agriculture and food processing resulting in a plague of health problems for modern man. Coronary heart disease, diabetes, cancer, osteoporosis, obesity and psychological dysfunction have all been scientifically linked to a diet too high in refined or processed carbohydrate. Search "Google" for Paleolithic nutrition, or diet. The return is extensive, compelling, and fascinating. The Caveman model is perfectly consistent with the CrossFit prescription."

Amy makes a good point that sometimes you are forced to make the choice to do something good now or to wait (possibly forever) to do something even better.

90% Paleo with a few dropouts is better than 100% Paleo with only a few finishers.

A pilot study that lacks a control group is better than no study at all.

A limited, single-center trial that wakes interest and paves the way for more complete work in the future is better than just not doing it because you can't do it perfectly.

Hi, some thoughts here.

It is interesting to me that the non-competitive approach worked well. In the CrossFit community, often competition is pushed. *Sometimes* we need to "step back" to basic science instead of arbitrary sport to make conclusions.

Any type of calorie restriction with any type of regular exercise leading to weight loss and better body composition/numbers is not exactly surprising.

Keeping track of workouts (and not just exercise, sets, reps, and time) is super useful, even without randomized, controlled, etc. etc., and the authors should be commended for it.

Justin

For total transparency, I also think the outliers should be listed with their relative outcomes. With N=15, these could be meaningful. Also, it would be really interesting on any future studies to include a detailed food log and to quantify the body changes relative to proportion of calories from different types of foods. "Paleo" to some is eating all spinach, kale, and salmon, while to others it can mean masses of bacon, brisket, and almond butter. Are people's body comps/biomarkers changing because of eating like the former, or in spite of eating like the latter? Obviously there is a lot of middle ground on which most paleo adherents stand, but these are questions that could be quantified.

Also, for anyone who is interested, searching PubMed for "paleolithic diet" or other keywords leads to several studies that study similar to questions as this one (not all of them are written by Cordain).

This article gives a lot of good info. I do agree with most of the others that you need a control group. I think you would actually need a few control groups though. You would need a paleo/zone group, a zone group/ a paleo group/ and a whatever you want diet group. You would have to break it into this to see what is actually causing the change if you are doing just paleo zone and a regular diet group you would not be able to see if the effects are from the quality of food or the quantity. For a good study you would need all four groups. These people would also need to follow all the same workout measures as far as workouts. They would have to be limited to 3 days a week so that everyone could do the same workouts. If you have one person working out 3 times and one person 5 it would blow the study as well.

First, I wanted to address the outlier issue. Identifying and excluding outliers is common in research with human participants. There are generally two ways to do this. One is to set a threshold (e.g., 3 standard deviations) for a particular dependent measure, and all data points that fall outside this threshold are considered outliers. The other way is to identify individuals whose experience was qualitatively different from the norm, for some reason. This is what happened with this study. One participant had been eating Paleo for two years before starting the study, so clearly he was atypical. The other participant identified as an outlier got sick during the study.

All the results were calculated with and without these two outliers. The findings did not change dramatically, but they did change a bit. For this paper, because it was geared toward a general audience, my primary goal was to represent the data simply, as averages - to show what were the typical results for our participants. Averages are skewed by outliers, and this skew can hide the real-world results that we're after. I was not trying to cherry pick or hide negatives, simply trying to represent the typical experience for a new convert to Paleo who was not sick. I did not include this explanation in the paper because the goal of the paper was not to definitively demonstrate the results, but instead to provide some food for thought and guidance for future research, and I wanted to avoid the science jargon as much as possible and keep it simple.

Second, I wanted to point people towards a list of published research on Paleo. I did not include a literature review in the paper because it is for a general audience and this would make it too long and too technical. I'm not pretending to be the first person to have approached this topic - obviously a lot of research has been done. I just wanted to explore the possibility of in-affiliate small-scale studies that cover a lot of measures.

http://www.thepaleodiet.com/published_research/

Finally, I wanted to point out that, while I appreciate the ideas set forth about multiple control groups, and I agree that follow-on studies that want to demonstrate causality need a control group, more is not always better when it comes to experimental design. The more groups you have, the more complicated and the more difficult it is to 1) actually accomplish the study and 2) interpret the results. Trust me, I have many years experience doing research with human participants, and it is way more feasible to do a collection or series of simpler experiments than one great big experiment with a ton of conditions. This is a hard lesson learned for researchers, as it is tempting to try to answer it all in one shot. But the best research is incremental.

And some affiliates that want to do this may not be interested in demonstrating anything, they may just want to try a different approach to the nutrition challenge, that will show their members clear before and after results. So they may not want to go to the trouble of a control group, and that's okay, this study approach is a great way to get buy-in and high motivation without pitting people against each other. Not to mention that our participants have stuck with it, because the structure the study provided enabled them to stick with it longer, and they saw the results in their numbers.

It would be good if some visual aid was provided in the results (e.g. plots). Something that lumps all the results so the trends can be observed. As an example. look at the HDL change: an average rise in 1 point with a max of 55 is hard to interpret because large ranges like that completely throw off the average.

Cool idea though, just thought the hard work would be given even more justice with all the raw data presented at some point (it's a small sample size after all), even if just at the end of the article.

Great to see this study Amy.
I also did a Paleo / Zone diet study, on NON CrossFit people - They undertook paleo eating for 6 weeks and mapped their results. Although we had a group meeting with 1/2 the participants at the beginning, I supported everyone by email only, some lived in other areas of the country.

I have outlined results here of 8 people, with participants' experiences in their own words (more people to be added).
By the way most kept up similar exercise. Some people switched from Zone to Paleo Zone. Some people switched from average healthy New Zealand diet to Paleo or paleo zone. All adhered to Paleo food choices fairly strictly. I wanted to see what the impact of making strict paleo choices would be for my Zone Clients who were all eating along Zone diet guidelines, but were including grains. Would the removal of agricultural foods make a difference to people who had already experienced improvements on the zone diet?

http://paleozonenutrition.wordpress.com/2010/09/01/6-week-paleo-trial-results/

A couple of things I did - which was interesting. I had people fill out a pre-trial meal diary so I could do an analysis and compare food ratios, calories and nutrients, and food choices.
I also had pre and post blood pressure - blood pressure generally improved, by 10 -20 points when high.
I also collected Vitamin d levels, just out of interest, as the study was mid / late autumn. Most people were low or borderline low to start, and many had further drops in Vit D even if they were taking supplements.

My overall conclusion - Paleo is better than Zone, Paleo food choices make a difference to those already following the Zone diet. My previous study - which I did in a gym some years ago showed that those who followed the Zone + exercise got much better results than those that followed a healthy recommended calorie restricted diet + exercise. I'm now convinced that paleo with or without Zone balance (Zone balance is great for getting a start on portion control and ratios for many - but I emphasize - it's a starting point) trumps the zone diet with any amount of non paleo foods. Appetite control appears to be so much better when paleo choices are all you eat, making the strict zone controls less required.

I would love to see further, more rigorous studies done with a variety of groups of people as mentioned in earlier posts.

I would like to see a systematic study of a paleo ketogenic diet. I'm currently doing this and have thus far been impressed overall. There isn't much data out there, especially with regard to fitness, so anything would be better than nothing.

Julianne, thank you for the link, that is great to have detailed case studies like that. Interesting results that definitely mesh with what I've seen.

Clinton, really good idea to investigate a Paleo ketogenic diet. Paleo can run the gamut from pretty high carb to almost no carb, so when we say Paleo, there is still a huge variable at large. Could be factored into an experimental design or amount of carbs could be tracked and factored into the analysis. Lots of ways to look at this, may not come up till sometime down the road but I'll definitely add it to the list!