[Marius_92 (OP)]

Hello everyone,

for our project we need a source for temperature data fulfilling following criteria:

1. we need historical hourly temperature data
2. a whole year has to be covered (ideally a recent year, like, e.g. 2019)
--> perfect would be a csv-file with 8760/8784 time stamps, each containing a temperature value
3. The data should be available for any geographical position worldwide
4. The data should be for free
5. The data should be allowed for the use within a commercial application
6. In the first step, one dataset for a specific location, downloaded manually is enough. In the long run, we need to get the data for many different locations with a python-function within our project

It is quite difficult, to find a website that fulfills all these criteria.
For instance, the NOAA site gets very close to our requirements. However, the data only covers the USA and it has to be requested and then gets sent per mail. We need, however, data for any location on the world, and we want to automate the process of getting the data from the website.
This site wunderground, on the other hand, only provides hourly data for a single day, but not for a year.

Does someone know a website, that fulfills ALL the criteria, where we can get these data from?

Thanks in advance!

[alancalverd]

Depends what you mean by "any location"!

Major airports certainly log a whole lot of very accurate meteorological data every 20 minutes but civil airports tend to be located only near cities and the military are less inclined to release data.

You are unlikely to find anything as detailed from any other source, and of course this excludes the 75% of the earth's surface that is covered by sea and at least 20% of the surface like polar and desert regions that doesn't have  permanent airfields. 

[Bored chemist]

If someone wants to build a wind farm they probably start by collecting data like that so, it's possible that if you talk to the owners of wind farms they might have it.
The problem you will run into is the combination of 4 and 5.
If you plan to make money, the data owner is going to want a cut.

If this was a school project, they might give you the data for free, but they are unlikely to be so generous if it's a commercial one.

If you accept that you are going to have to pay, you might as well talk to people like the Met office.

[Marius_92 (OP)]

Thank you so much for your quick responses and your tips!

@alancalverd:
You are right, "any location " is not very exact. Sorry.
We need it for a literally any location, where you could build an industrial or other business building.
this includes almost all land surface, but excludes seas and also poles. Most interesting is of course land surface, where human settlements are.
The tip with the airports therefore might be sufficient for a first, less precise approach, or as a fallback solution, in case we can't get anything more precise. the optimum would be, if some company gives us the location of their building, and we can find the temperature values for the exact location, rather than those of the next weather station.
However, there is another issue with the airport-approach, which addresses the issue of scalability: We want to automate the scraping of the data. So it would be good, if all the data could be accessed from a single page. It is not feasible for us, to first search the next airport and then the website of this airport, and then check, if there is any data, each time we want to use our application

@Bored chemist:
unfortunately are all the points mentioned necessary. If we cannot find a datasource, that fulfills all the requirements, we have to go another way. A non-free solution is no option (this is a requirement from the company, not from me)

[Bored chemist]

Data is valuable.
If you succeed in finding someone who gives away free money, please let me know.

[Marius_92 (OP)]

Thanks for your useless answer. To be honest, I don't get your point. There are tons of free data on the net on all kinds of topics. Even for temperature data there are hundreds of sites. I just found no site so far, that provides exactly the data that I need in the correct form.

[Marius_92 (OP)]

Hello Everyone, we are still looking for the specified website/datasource.
To this comment, I attached an example, how the data should look like. Maybe this can help you get an idea of what we are looking for.

[alancalverd]

Two reasons you won't find it

(a) nobody knows it and if they did

(b) it's as commercially valuable to them as it is to you, so they won't let you have it.

However as good engineers you know what to do: take the best available (airport METARs), use sensible correction factors for terrain and altitude (I think "Meteorology for glider pilots" is still in print, and very readable) and get your lawyer to write the contract so it isn't your fault if it all goes wrong.

If you just want to design a building, talk to a local architect. You'll need to do that anyway to get permission and negotiate with infrastructure and builders. There are plenty of "green" architects around who know how best to exploit the local environment, and plenty of industrial consultancies who can do the structural calcs to hold it all together.

[Bored chemist]

To be honest, I don't get your point.

Are you now starting to understand my point?

[Marius_92 (OP)]

To be honest: no XD
Both of you didn't help me even a little bit XD
Just because you don't know a site, that provides the requested data, that doesn't mean that it doesn't exist.
But I have the feeling you two enjoy yourself talking as if you were smart XD

[alancalverd]

 Not smart, but a world-weary  project manager, old aviator, and businessman.

A couple of years ago I was playing in a pickup jazz band with a guy from a German bank. Usual chat in the tea break about our day jobs. Turned out that he handled small capital project finance. I said "Every project I have seen in the last 40 years has run 10% over time and 20% over budget"   He said "Yes, we use the same figures in Germany".  I know what I'm talking about.

[Bored chemist]

Just because you don't know a site, that provides the requested data, that doesn't mean that it doesn't exist.

But that's not what we did, is it?
Try reading the posts again.
Do you see where we wrote

it's as commercially valuable to them as it is to you, so they won't let you have it.

and

Data is valuable.
If you succeed in finding someone who gives away free money, please let me know.

Do you see that we didn't just say that you wouldn't find the data.
We explained why you wouldn't find it.

So we explained that you should stop wasting your time looking for something that's impossible.

Do we need to explain it again?
Is there a grown-up there who helps you with things you don't understand?