Turning machines (and finite feed-forward ANN) cannot solve the halting problem. Our minds can do it...I agree that our minds can solve simple cases of Turing's Halting Problem.
The huge effort put into teaching Structured Programming over the past 40 years or so is aimed at limiting people to generating the small subset of software for which the author (and their colleagues) have a chance of understanding its structure, and visually solving the Halting Problem.
Given well-structured software, a good optimizing compiler can also "understand" the structure, and solve the Halting Problem.
But given a really convoluted piece of code, perhaps requiring solution of a unsolved mathematical problem, no human or computer can resolve the Halting Problem.
Example: Input a positive integer. If it is 1, Halt; if it is even, halve it; if it is odd, triple it and add 1.
Question: Does this Halt in a finite time? (ie Turing's Halting Problem)
Answer: Nobody knows (last time I checked).
You can check specific values:
Some numbers (like 64) halt very quickly.
Some much smaller numbers jump around erratically for a very long time (27 reaches as high as 9232).
But the Halting Problem asks about every possible input, and we simply don't know.
So I think that solving the Halting Problem is a false dichotomy between humans and machines.
PS: I hope I have recalled this simple example correctly. But the point is that even simple questions in computation can have an unknown outcome.
...Some software gurus have even suggested banning simple operations like "triple it and add 1", because they make software impossible to analyze!