[varsigma (OP)]

With all the fuss about information entropy having the same kind of equation as thermodynamic entropy, and information theory being about how much structure or hierarchy there is in ah, the system under consideration, I've got a question.

My question for the pundits is, how much information is there in a scientific theory? Can it be compressed, transmitted and received, this information, which would be a necessary thing for it to "be" information . . . ?

But, is the question meaningful? If it is then it must be about information, per se. If not, . . .

[alancalverd]

"Information" presumably describes the actual state of a bounded and defined system - e.g. the use of information theory in speech compression.

A scientific theory  describes the hypothetical behavior of a system that may be described (two bodies colliding) but isn't defined or bounded (could be any two bodies, anywhere in the universe).

[Eternal Student]

Hi.

My question for the pundits is, how much information is there in a scientific theory? Can it be compressed, transmitted and received, this information, which would be a necessary thing for it to "be" information . . . ?

    Science is a big area.   A theory in Biology, like Darwin's theory of evolution, is a collection of words.   It would seem to be information just like any other collection of words conveys information.
    I can't think of any area of science that doesn't require some words to go with it.   You stated yourself in another thread.....

....... after a bit of thought, the idea that a scientific theory can be reduced to a set of equations is perhaps an idea of simplification, but I don't know that that can be true:  without some contextual language a page of equations can't convey that much. Suppose Einstein had *simply* written his famous formula (a system of equations!) for gravity in one line and published it.

    Furthermore, the use of equations is also just some symbolic form of communication, a written language.   For example,    x + 2  >   4            conveys the same information as   x > 2    but the second form seems to have less characters and neither of them makes any sense to anyone unless they've been taught about these symbols by someone else during their life.   There are also clearly rules, like rules of grammar and syntax in other languages,  you couldn't shuffle the symbols into any old order you wish....   4 + 2 x >        doesn't mean very much at all.   Mathematics is a language, it may also be other things as well, but it seems certain that it is partly a language.

Best Wishes.

[varsigma (OP)]

This question I'm asking might be nonsensical, or it might be just really hard to think about.

One thing about information, you do need to tie it down and consider a subset of the many forms of information we know about.

So instead of a theory, maybe consider a set of equations. Or, consider how some equations don't change their form when physical quantities change. An example is the equation for entropy, another is SHM. Can anyone name any other equations whose form appears to be independent of the physical context (?)

With entropy, if you start with the Shannon version then just multiply it by Joules per degree Kelvin, since this actually is dimensionless; it makes a lot of sense that the equation spans information theory and thermodynamics. It says energy and temperature are things we concoct to explain this, and have a thermodynamic basis for entropy. But that's about it.

[alancalverd]

As far as I am concerned, "information" is only meaningful if it applies to a specified system, whilst an equation is not specific. Same difference between a definition of momentum = mv and the actual momentum of a moving car - only the latter is information.   So "theoretical information" either means a test input (as distinct from real data) to an information processing system, or nothing at all.

[varsigma (OP)]

As far as I am concerned, "information" is only meaningful if it applies to a specified system, whilst an equation is not specific. Same difference between a definition of momentum = mv and the actual momentum of a moving car - only the latter is information.   So "theoretical information" either means a test input (as distinct from real data) to an information processing system, or nothing at all.

I agree that it's necessary to specify a system.

I think that equations of motion, for instance, are things you can write as a computer program. Which is something that's been done for a while. But the idea that the motion, the momentum, of a real object can be represented by an equation and that this equation, since it can be a "data object" in a computer program, is saying something deeper about physics and simulations of it.

Which is something along the lines of, really it's all just information and representations of it. The closure is that real objects in motion are just "real, physical" equations, a universal program algebra if you will. We know some of the algorithms, but also that there are many we don't yet understand--we haven't characterized the data structures satisfactorily. We don't even know what the "no-operation" or identity instruction is.

If, that is, all the information we can talk about (or represent) is something the universe is computing for us, but not just us (that would be weird).

[Eternal Student]

Hi.

LATE EDIT / WARNING:    Halves    1/2   aren't displaying properly.    ?at²   is    one half of at²
 

    I probably don't know enough about Shanon's information theory.

Can anyone name any other equations whose form appears to be independent of the physical context (?)

   What about  2 = 2     ?

More generally:
Let's assume that most of physical science is about describing relationships between two or more things we can measure.
    For example,    s = ut  + ? at²    describes a relationship between these things:
  distance travelled (s) ;    time elapsed (t)    ;  the (constant) acceleration (a) of an object.

So we might assume there's a bucket of things that we can measure,  for example we could measure the mass of the object, the spectrum of light reflected by the object,  the acceleration of the object,  the time elapsed,  the temperature of the object, the number of petals on my rose   etc.

     We could always perform multiple variable polynomial regression between the thing we're interested in  (the distance travelled by the object) and ALL the possible variables that might be related to it.    So we know that.....
    s =    some polynomial in t    + some polynomial in a + some polynomial in (number of petals on my rose) + some polynomial in the cross-terms  (like  at,   at²,   etc.)  + .........  + (some error terms in the statistical model)

    After-all, if one or more combinations of the variables aren't important   (let's say the number of petals on my rose and the temperature of the object) then we'd just have 0 co-efficients for those.

    What this particular scientific lemma or equation (  s = ut + ?at² )   is telling us is that, provided we have constant acceleration.... and ignore a bunch of other things (air resistance and most of real life).....  then we can set most of the coefficients to 0 and will only need the terms    ut  and  ?at²  in our statistcal model   for the best fit regression model.    It would actually have 0 valued error terms so be a perfect fit..... provided we do ignore air resistance and the bunch of other things that appear in real life.

    Anyway.... an approach something like this may indicate how much Shanon information is provided by a typical equation or simple lemma in Physics.    It's just a thing that indicates how we should set the coefficients of the variables in a multi-variate polynomial regression compared to all the possible ways  (a priori) that we could have set those coefficients.

    So, the   SUVAT  result  discussed above   (s = ut + ?at² ) has a high information content if you orginally knew nothing about Newtonian mechanics because it sets a lot of coefficients for things that might have been relevant to 0.     However, if your prior knowledge was all of Newton's 3 basic laws of motion then the information content of the equation   s = ut + ?at²     is  0   because it tells you nothing that you didn't already know.   You could have derived the result from Newton's laws and so you already knew what all the coefficients were going to be in your multi-variate polynomial regression model.

    Human beings tend to regard a scientific theory as being more useful if it involves only a few variables and care a little bit less about some non-zero error terms that might still remain.   But I suppose there is some preference to have small enough error terms for a really usefull theory.    Anyway, a theory in physics that says   "the curvature of spacetime is related to some stuff and not related to some other stuff" contains 0 Shanon information because we could already assume that was the case.  A theory that says   " the Einstein Tensor  G_μν  =  k.  T_μν" has a much higher Shanon information content because it actualy does allow us to set the coefficients for a load of stuff to 0 in our multi-variate polynomial regression and tells us that we only need to consider the stress-energy content of our space.

     The value of a scientific result (perhaps an equation) to a human being isn't going to be determined just by its Shanon information content.    s = ut + ?at² may have 0 Shanon information content if we knew Newton's laws to start with but it's still very useful to have that resut to hand instead of having to drive it from first principles every time.

(I don't know.... just an idea or one way to consider things).

Best Wishes.

[alancalverd]

What this particular scientific lemma or equation (  s = ut + ?at2 )   is telling us is that, provided we have constant acceleration.... and ignore a bunch of other things (air resistance and most of real life).

This equation is a statement of absolute fact, not an approximation or idealisation.

In practice you do not know any of the parameters absolutely, but you can't set aside the mathematical fact that  distance covered is the integral of the velocity vector over time. It has nothing to do with Newton.

[Eternal Student]

Hi.

you can't set aside the mathematical fact that  distance covered is the integral of the velocity vector over time.

    Yes, it probably wasn't the best example I could have used.   I just wanted to cite an equation that was familiar to most people.

This equation is a statement of absolute fact, not an approximation or idealisation.

   It requires a constant acceleration, a non-expanding universe and other things  - but it's almost an absolute fact.

Best Wishes.

[alancalverd]

Admittedly it is a special case where a is known constant but for the purposes of rugby football, motor racing, aviation and space travel we can use the general  ∫at dt to find out where we are at a given time - it's the basis of inertial navigation.

In an expanding universe I guess all the base vectors expand together so the equation still holds.....

My point is that any actual numbers you use for u, a and t constitute information, but the equation doesn't: it's the process by which we extract the information s. So you could say it's a lossy compression algorithm!

[Eternal Student]

Hi.

My point is that any actual numbers you use for u, a and t constitute information, but the equation doesn't

    Maybe that is what you and @varsigma want to discuss or consider, which is fine.  You do as you wish.

    It's just not something that is required or stipulated when defining the Shannon information content of a message:


information definition.jpg (54.37 kB . 802x366 - viewed 499 times)

Definition taken from    https://www.umsl.edu/~siegelj/information_theory/information/shannonsdef.html

The OP included this question:

My question for the pundits is, how much information is there in a scientific theory? Can it be compressed, transmitted and received, this information, which would be a necessary thing for it to "be" information . . . ?

... and one possible way to consider the information content of a scientific theory is as outlined earlier:
    Suppose you originally thought   s = ut ;   s = 0 ;    s = (1/2)at²      or   s =  ut + (1/2)at²  would be the right relationship,  with equal probability for each of these.
   So you had 4 possibilities with a probability 1/4  for each.
     Then you're given the equation   s =  ut + (1/2)at²
    So  I(E)  =  - Log₂  (1/4)  =   Log₂  (4) = 2.
     The Shannon information content is  2  (they tend to use Log to the base 2 and then declare the value of I(E) to be in "bits",  2  bits of information have been learnt). 
     
    Suppose we already knew that   s =  ut + (1/2)at²,    So the probability for this being the relationship is 1,   Then we are given the equation   s =  ut + (1/2)at²,
    So  I(E) =  Log₂ (1)  = 0   and we gained 0 bits of information.

   I can't see anything in the definition of Shannon information content for an event that demands u, a and t  had to be actual numbers for a particular object like my bicycle on the journey to the shops this morning.   Why not apply it to the information content provided by a scientific theory (or even just a single equation that appears in that theory) that specifies one particular equation is most appropriate out of a whole set of equations that might have been conisdered appropriate a priori?

     I'm not any sort of expert in Information theory.   It just seems like one way you might be able to consider the information content of a particular scientific theory.

Best Wishes.     

[alancalverd]

Suppose you originally thought   s = ut ;   s = 0 ;    s = (1/2)at2      or   s =  ut + (1/2)at2  would be the right relationship,  with equal probability for each of these.
   So you had 4 possibilities with a probability 1/4  for each.
     Then you're given the equation   s =  ut + (1/2)at2
    So  I(E)  =  - Log2  (1/4)  =   Log2  (4) = 2.
     The Shannon information content is  2  (they tend to use Log to the base 2 and then declare the value of I(E) to be in "bits",  2  bits of information have been learnt).
     
    Suppose we already knew that   s =  ut + (1/2)at2,    So the probability for this being the relationship is 1,   Then we are given the equation   s =  ut + (1/2)at2,
    So  I(E) =  Log2 (1)  = 0   and we gained 0 bits of information.

But I could equally have thought s = 2/3at², s = (v - u) t, or even something that doesn't make dimensional sense (yes, I've marked physics exam papers)..... So you are asserting that the information content of an equation I(E) is undefined unless we already know it, in which case I(E) = 0. 

I think this supports my argument that a scientific theory, at least if expressed as an equation,  is not information since, if correct, it doesn't inform the receiver!

[varsigma (OP)]

What I think I might be trying to look into here is something called duality.

It's a common thing in physics to have dual representations of . . . physical realities.
This is an apparent thing in the formula for entropy. Here, one could argue that the form of the equation is preserved when moving or transforming from a thermodynamic to a strictly computational basis, because a duality exists which constrains the representation. This is assumed to be universal, any alien scientists would be similarly constrained and would encounter the same duality, hence would form the same type of equations, in an alien language of course.

As to the information in a theory being compressible, transmittable and receivable, that seems to me to be a requirement if the idea is going to fly.
If you write an equation of motion on a fixed background, how is that different to writing code that defines the same equation as a data object? The encoded version can be a dynamic thing, right? One could propose that the data object is going to follow an abstract path through a "computational" graph. From which we get dual representations. One is a fixed equation, the other is a moving object.

So if there's a chink in the wall here, we see a duality in the wave-particle model(s) because of a constraint or restriction (or maybe it's really an obstruction of an embedding of ... something). Something to do with observation and measurement, both clearly in the transmission/reception of information domain. What is the codomain?

ed: in my theory, such as it is, Shannon entropy is a measure of fixed amounts of information--fixed length messages; whereas algorithmic entropy (a la Chaitin) is not fixed the same way. There is some "crosstalk" between entropy and complexity, or maybe some interdependence.

ed2: I wanted to look at the formulation of SHM in particular, because of the equivalences between mass in the Newtonian picture (which has an alternating kinetic and potential energy), and inductance in the Maxwellian one, which considers alternating (oscillations of) electric and magnetic potentials. What is this equivalence telling us about wavelike motion, one in a gravitational setting, the other in an electromagnetic one, far from any gravitational influence because of the difference in coupling strengths?

I mean you do need a lot more Newtonian mass than you need electric current to see the same energies, but the duality doesn't depend on that, fairly obviously.

[alancalverd]

So if there's a chink in the wall here, we see a duality in the wave-particle model(s) because of a constraint or restriction (or maybe it's really an obstruction of an embedding of ... something).

Common misconception. There is no "duality" inherent in the phenomenon, but we need two distinct models to describe different aspects of it.

Apropos the equations of motion, or indeed any analytic mathematical statement,  I hold to the view that it doesn't in itself convey any information, but when you try to simulate it in either an analog or digital computer you create information in the particular system that will carry out that simulation.

[varsigma (OP)]

Common misconception. There is no "duality" inherent in the phenomenon, but we need two distinct models to describe different aspects of it.

That's one argument--there is no true duality. What there is is measurement, and, we know how to make an interference pattern appear, and how to make it disappear, with measurements.

But, any quantum interference experiment can be formulated the same way. It's something like how the same formula applies to any object in free fall, say.

Experiments, with say light, currently still haven't shed much on the answer to whether this is a true duality.
There are experiments that slow light down, in condensates, and apparently photons have been captured and released, in these things. The catching and releasing involves energizing the condensate with tuned laser pulses. So photons are caught and, I've seen an image of the shockwave produced by the interaction. What kind of information is (in) this image?

more: I've recalled that the interference effect and making it appear or disappear, is a matter of having a continuous beam of quantum particles incident on a double slit diffraction grating. Then you can measure which-way information by counting particles at one slit, which "erases" the interference fringes. There is a duality here though, because you can shift the phase of the interference fringes, with a 1/2-wave plate at one slit, or with a solenoid between both slits (oops that only works with fermions). Both methods of shifting the pattern of fringes demonstrate a gauge symmetry. The gauge is the shift in the pattern, after measurement.

[alancalverd]

, we know how to make an interference pattern appear, and how to make it disappear, with measurements.

Absurd, if commonplace, statement!

Time moves in one direction. If you observe an interference pattern, it must have been formed somewhere in space and time before you observed it. How did the wave know it was going to be observed?

If you modify the physics of one slit, i.e. "observe" particles passing through it, you obviously can't expect to see the same resulting pattern.

It all comes back to the fact that we need two models: a wave model describes propagation, a particle model describes interaction. They are both models of the same entity, but that doesn't imply that the entity has any inherent dualism.

"Duality" is a human conceit, not a natural phenomenon!

[varsigma (OP)]

Absurd, if commonplace, statement!

Time moves in one direction. If you observe an interference pattern, it must have been formed somewhere in space and time before you observed it. How did the wave know it was going to be observed?

If I observe an interference pattern after performing certain "experimental" actions, there it is, say on a wall.
Anyone can produce such a pattern, but in a "properly equipped" physics lab, you might encounter an optical setup with lots of precision in it. Wavelengths are known, slit dimensions are known. Distances between the various devices are known.

It's also well-known that an interference pattern can be "destroyed by measurement", on one path. That's established science.

[varsigma (OP)]

If you modify the physics of one slit, i.e. "observe" particles passing through it, you obviously can't expect to see the same resulting pattern.

How does that explain the way fermions, like electrons, in a two-slit experiment can be phase-shifted? The solenoid between the two apertures,  the Aharanov-Bohm effect, the shift in the pattern. How does your "obviously can't expect" fit in there?

In that, the effect was predicted but it took until the devices, necessarily in the very small domain, could be made. Aside from that, bosons don't have this symmetry, they aren't charged particles. But I can produce an interference pattern with a laser pointer and some lead pencil refills.

[alancalverd]

As I said, if you modify the physics, you get a different pattern.

When you say "wavelengths are known" you actually mean that wavelengths have been assigned to the wave model that predicts the spatial distribution downstream. But when you detect the photons/electrons/whatever downstream of the slits, they turn up one-by-one with the same energy as the source, i.e. the interaction with a detector follows a particle model.

The problem with "duality" is that it presupposes classical continuum mechanics to apply to all phenomena, then invokes magic to explain what actually happens. Far better to start with the quantum observation and state that we use two distinct mathematical models to describe propagation and detection.   

But I fear we are drifting off-topic, as usual!

[varsigma (OP)]

When you say "wavelengths are known" you actually mean that wavelengths have been assigned to the wave model that predicts the spatial distribution downstream. But when you detect the photons/electrons/whatever downstream of the slits, they turn up one-by-one with the same energy as the source, i.e. the interaction with a detector follows a particle model.

Aren't you just invoking a duality between a photon "wave-model" and a photon "detection model"? Do you think it's possible that photons can be observed as both, or as neither? I can point to some experiments.

The problem with "duality" is that it presupposes classical continuum mechanics to apply to all phenomena, then invokes magic to explain what actually happens. Far better to start with the quantum observation and state that we use two distinct mathematical models to describe propagation and detection.   

I would say one problem with dualities in physics is finding reasonable explanations for them. Why is the electromagnetic field an example of self-duality?

When a laser beam interacts with a beam-splitter, is there a detection or a measurement? Or when it interacts with a double slit?
In what sense is emitting a photon detecting it? What is detected or measured by a pair of slits? Is it just that we need to have a duality, even if the math says it's all the same?

ed: correction I should say the math says quantum states are conserved, but not by measurement. Unfortunately then, the math can't say what measurement is. But wait, we seem to have to make the quantum states interact with large quantities of matter, on a per-particle basis to claim we have a measurement.