# The Naked Scientists Forum

### Author Topic: Is there a better way to calculate trend lines on periodic functions?  (Read 4352 times)

#### CliffordK

• Neilep Level Member
• Posts: 6321
• Thanked: 3 times
• Site Moderator
##### Is there a better way to calculate trend lines on periodic functions?
« on: 06/03/2011 04:48:47 »
Actually looking at some of the environmental information.
It seems as if there is a problem with poor fit of trend lines on periodic, or partially periodic functions.

I think some kind of a least-squares trend line is typically used.  These calculations are based on the Slope() and Intercept() functions in OpenOffice.

To illustrate some of the problems, here are a couple of basic functions.

Plotting a trend line to a sine wave.

If I plotted for 0 to 1800 degrees (5 complete cycles), starting on a peak and ending on a valley, I got a really bad match (orange line)
I got a reasonably good match for the slope, but was off on the average/intercept if I plotted from 0 to 1620 (5 peaks, 4 valleys), or 180 to 1800 (4 peaks, 5 valleys).
The best match was plotting from peak to peak or valley to valley.  (90 to 1530, or 270 to 1710).

Anyway, so choosing the end points is very important.

Then, I decided to try a sawtooth function (up slope, 10/8, down slope -10/2)

It turns out that it is very hard to get a good match for the trend line (which should have 0 slope, average of 5).

With the exception of starting on a peak and ending on a valley, the majority of the trend lines have a positive slope.

I suppose my problem is that if can't match a trend line to a simple geometric function...  how can I trust matching it to natural systems.

Suggestions?

#### Bored chemist

• Neilep Level Member
• Posts: 8591
• Thanked: 41 times
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #1 on: 06/03/2011 10:45:13 »
Do you know about Fourier analysis?
http://en.wikipedia.org/wiki/Fourier_transform

#### CliffordK

• Neilep Level Member
• Posts: 6321
• Thanked: 3 times
• Site Moderator
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #2 on: 06/03/2011 21:28:30 »
Do you know about Fourier analysis?
http://en.wikipedia.org/wiki/Fourier_transform
Thanks,

Yes, I know about the FFT...  at least in theory.
Perhaps I should look at it some more.

What I'm actually looking at is the Temperature Trends.  In particular, 1997/1998 to present.

As I noted from the sine curve above, one can obviously diddle with the end-points to prove anything that one wants.

Visually looking at the graph, it appears to have no significant trend.

But, when I plot the trend line, it invariably gives me a slight positive slope

What I noticed was that the temperatures from 1999 to 2002 and 2008 to 2010 seem to increase in a sawtooth pattern, with a sharp downswing, and a gradual upswing, thus weighting more negative towards the early years, and more positive towards the later years.

And, what I was able to demonstrate with the stylized sawtooth pattern is that it will cause an upward trend in the trend  line in most cases except when one begins with a peak and ends with a valley.

If I was looking at something with a slope of 1:2 or so...  I really wouldn't care much about a fractional difference.  But, with a nearly flat slope, it makes a huge difference.  [xx(]

Anyway, it doesn't seem like a FFT type of function, but perhaps it will give me additional analysis tools and it certainly is something that I need to understand better.

#### Geezer

• Neilep Level Member
• Posts: 8328
• "Vive la résistance!"
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #3 on: 07/03/2011 05:03:46 »
I think BC is right. Try breaking it down into frequencies and see what the relative amplitudes look like. I have a suspicion that we humans can "eyeball" just about anything that takes our fancy from complex waveforms. (I've done a lot of that.)

#### Bored chemist

• Neilep Level Member
• Posts: 8591
• Thanked: 41 times
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #4 on: 07/03/2011 06:58:31 »
If it was a sawtooth then an FFT (or any other FT) would show it very clearly.
I don't think you can say it's real on that data.

#### imatfaal

• Neilep Level Member
• Posts: 2787
• rouge moderator
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #5 on: 07/03/2011 13:56:34 »
Cliff - BC or one of the other practising scientists can guide you on this but IIRC a regression coefficient R^2 of 0.004 is far too low to allow the correlation to be claimed.  For a simple linear regression R^2 ranges from 1 to 0; with 1 being a complete correlation (ie y=ax+b) and 0 being no correlation.

This is quite apart from the fact that linear regression is the wrong tool - but is does explain why you were mystified to see the slope, it's only just evident by the slightest of margins!

#### Bored chemist

• Neilep Level Member
• Posts: 8591
• Thanked: 41 times
##### Is there a better way to calculate trend lines on periodic functions?
« Reply #6 on: 07/03/2011 19:26:58 »
A full statistical analysis s a lot of work, but there's one simple thing you can do.
The value of R^2 gives you the fraction of the variation that is accounted for by the "model". So, in this case, you are assuming a model that says the data fits to y=mx+c and has noise that explains why it doesn't all lie exactly on the line.
In this case, the model explains 0.0049 of the variation in temperature and the other 99.5% of the variability is caused by something else.

As Imatfaal says, that's pretty hopeless.

#### The Naked Scientists Forum

##### Is there a better way to calculate trend lines on periodic functions?
« Reply #6 on: 07/03/2011 19:26:58 »