I have learned a good lesson over the past 7 months. There are good ways to experiment, and not so good ways. "Experiment", as I understand it, means to do a test or tests, from which some information can be gained. But, before any information can be gained, certain preliminaries are necessary.

If we are going to experiment with concentrations of phosphate in our water, for example, the very first thing we have to do is establish that we can measure how much phosphate is in our water. And, that means lots of calibration of the system used to do the measurements.

If we are going to experiment with various substrates, or techniques for growing aquatic plants, which was my purpose 7 months ago, the first thing I should have done is "calibrate" the equipment I was going to use. I skipped that step.

I found a very good deal on a 10 gallon tank, then assumed that because I had been able to grow aquatic plants well in a 29 gallon and 45 gallon tank, and less well in a 120 gallon tank, that growing those plants in a 10 gallon tank was a given. Now, 7 months later I realize that until I actually did grow aquatic plants well in the 10 gallon tank, I couldn't assume that I knew how.

Tom has repeatedly pointed out the need to do testing in several identical tanks, first establishing a baseline. For example, first establishing that you can, at will, grow plants algae free. Only then can you begin to test various methods for growing plants algae free.

So, my first effort with that little 10 gallon tank should have been a pressurized CO2 tank, with high light intensity, using the same SMS substrate I knew I could make work well. That would be the "calibration" of the tank. Since I didn't do that all of the "tests" I tried for 7 months were just playtime. Of course playtime is fun, and I did have fun, so it wasn't a failure, except that all I really learned was how not to do a test.

So, I'm passing on that lesson.

"Pain" is a good teacher.

Ah Hoppy! It seems like this 10 gallon tank will be haunting you for a long time.
Don't worry, I will soon be joining you in your misery =)

Another simple method/idea along these same equivalent is to use a "control", reference, or as Vaughn mentions, a baseline.

You need some point of reference to compare your results against.
Aquariums are not easy systems for many hobbyist to make similar to each other.

I use non limiting approaches to rule out things like low PO4 or CO2 etc.
Then I know that growth is not limited due to these other factors.

Only then can I limit and manipulate the variable of interest and get a good handle on what is happening with much confidence.

Folks sometimes wonder how I know, or exudes all this confidence, and am so aggressive in my argument when I read what I know to be a myth.

Heck it's not personal, it's just I've done this and bothered to think about it, test it, made the mistakes and came to a very different conclusion based on the evidence I found.

Not what I want it to be
Some seem to have a fore drawn conclusion based on some idea they read.
Or are not honest about their steps.

Vaughn is being honest and humble.
That is a key for any good experimenter.
More than anything I'd say. You realize how little you do know and are not going to try and make more out of the test than is there.

Yes, many folks get excited and think they found the golden fleece............then only later do they realize that it was not the case and it was something else folks had mentioned all along.

Things like CO2 will get folks even 10 years into the hobby.

One thing that I've been considered is using a different type of test method, more based on observation on systems I cannot manipulate and control. We have little choice in many ecological situations, and we need to draw inferences from lakes, rivers etc. Many aquariums also fall into this group.

But suggesting cause and effect in such systems is very hard at best, there are correlations and models to try and requires a lot of statistical keeness. And I've yet to see any aquarist use this approach.

You also learn how to be a good grower in many tanks, situations over time.
Once you get there, then you can test things much better.

You have more control.

Regards,
Tom Barr

May I also suggest that a Very Good Idea™ is to write down a "mission statement" of exactly what you want do, and how you are going to go about doing it (in detailed steps). Then make notes while doing it. I've had to force myself to do that quite often, as "mission creep" begins and I end up with nothing to show. (This goes for just about anything, not just aquaria-related projects.)

-Jason

As Jason mentions here,

A goal about what you hope to answer in your query is always a wise idea.
I read virtually every week or so, some well intentioned aquarist trying to set up some test, or read some reference/s but then not using a test method that will answer their question/s they hope to answer. The road to Hell is paved with good intentions.

You must have a test that answers the question you pose.
You can pose every sort of grandiose question you want, but the limiting issue is WHAT you can practically answer with the resources you have.

That's the bottom line.

While many aquarist certainly are well intentioned, often times the results and the claims without looking into it critically can lead to far more problematic myths and other poor conclusions.

I get in a funk when I see this and complain, but rather than seeing the errors, some assume it's a personal attack. I've tried to tip toe around folk's ego's in the past, but it rarely did any good. They got offended anyway.

I do not get offended, I get worried and re check things to make sure I did not put my foot in my mouth, which I do often do. Then I correct it and try not to do that mistake again.

Vaughn's hind sight shall serve him well.
After being burnt a few times, you learn.
But you have to get burnt and realize it and not be so darn certain, sure of what you did and make sure to go step by step to rule out each assumption as you go through things.

We are feeble minded, more than we realize................so we need Science and other logical progressions to figure much out.

What are the "safest bets" basically.
There is no absolution however.

Regards,
Tom Barr

Back when I worked as a research engineer we often followed the flow when we did testing - we would be looking for an answer to one question, but the testing pointed in a different direction, so we were soon off on a different quest. That is an effective technique if you have the time, money and equipment to do that. But keeping an open mind is always the most important part. And, accepting that you were wrong about something is another important part.

That latter point trips up even professional researchers. We get wedded to our great ideas and it is hard to "annul" the marriage when it is appropriate to do so. The natural tendency is to adjust our theory, do more testing, fudge some results, get mad at someone, but only at the very end actually accept that we were wrong.

Yes, but you must give up even the best model/theory if you falsify it.
I think that is more interesting. When I realize something does not work, or is wrong, then it's a good thing.

Then I know I cannot induce algae with PO4, Fe, NO3 etc.........

I still do not know what induces all algae, or every possible cause for algae, but I know what something is not along the way. I have some notions about what might cause algae, but there are always possibilities and likelihoods that there may be multiple causes...........

A typical test will involve a null and an alternative statement:

Ho: that there is a difference in algae blooms, presence/inducement at 0.2ppm, 1ppm, or 2 ppm of PO4.
Ha: that there is no difference in algae blooms, presence/inducement at 0.2ppm, 1ppm, or 2 ppm of PO4.

You can test the PO4 levels, and the algae presence.

These are very specific statements and one of them cannot be true.
I used Paul Sears' notion about algae and PO4 to test.
My alternative hypothesis was that there is no difference.

The questions posed are not geared at who is right or wrong, it's gears only at the set of questions.

Another researcher might repeat it and find that there is a significant difference and accept the Ho/null.

However, did they control for NH4, fish load, CO2, light, current, plant species type, general biomass in the tank etc?
More often than not, the answer is no, they did not.
Scale and time for the responses to occur also play roles.

There are so many things we do not think of and overlook.
And when you have considered many of these things and account for them, ask others about them, acknowledge them, then it shows you have done your homework.

If not, well............