[sazr (OP)]

Do we know what sort of preprocessing the retina does? And more importantly what the output of this preprocessing would look like? I'm interested from the perspective of computer vision uses, I can copy what the retina is doing to efficiently pre-process (and remove redundant information) and perform higher up processing that the brain would do.

If anyone could describe visually what the output of retinal pre-processing would look like that would be extremely helpful. For example, would an example output look like just edges (so black wheres theres no edges and white/colour? where there are edges). I am interested in any/all eyes that have retina's not just humans.

[chris]

This is quite a tricky question to answer. There is a lot of retinal processing across the multiple layers of the retinal; these include steps to sharpen edges and reinforce contrasts, ways to enhance or bolster colour vision under low-light conditions, ways to electrically disconnect rods from vision during high light intensity situations, surround inhibition to improve acuity, and so on.

People making retinal prostheses are getting quite good at decoding retinal processing so that they can inject signals back into surviving components of the optic nerve, but it's very much early days and we're pretty much at - literally - the black and white tv with very few lines stage in our understanding at the moment.

[evan_au]

There is not a "one size fits all" version of retina processing.

Like all neuronal cells, there is a critical phase of learning where they are exposed to environmental stimuli, and the inputs they receive during this period help define the connections they keep for later in life.

The "spike" signaling used by neurons (with 1 or a few bits of information) uses very different "algorithms" than the fixed and floating-point numbers we use for image analysis (with 12 to 32 bits of information). It also appears much more energy-efficient; some companies are developing neurone-inspired computers that can work with many 1-bit or 2-bit values in parallel, or to use "spike" signaling (which is mostly idle).

[sazr (OP)]

these include steps to sharpen edges and reinforce contrasts, ways to enhance or bolster colour vision under low-light conditions, ways to electrically disconnect rods from vision during high light intensity situations, surround inhibition to improve acuity, and so on.

@chris Thanks this is exactly what I am looking for. Do you know how the retina is; reinforcing contrasts, bolstering colour vision, doing surround inhibition? Ie, what the retina filter techniques/algorithms are? I want to try to emulate them in code and create my own crude retinal filter/retina processing. Maybe you can point me to literature to better understand how the retina performs these 'filters'/processing?

[evan_au]

what the retina filter techniques/algorithms are?

This is an experiment where they connected electrodes in a monkey brain into an AI system which generated random images until they found precisely what each monitored neurone was "looking for".

This is recorded deeper in the brain than the retina, but it is a general-purpose learning mechanism that could be used to learn the function of neurons in the retina.

See & listen: https://www.sciencefriday.com/segments/neuroscientists-peer-into-the-minds-eye/