Apparently I didn't lose my massive wall of text. I hope I addressed your key points, it makes sense, and that it's not too long.

Caveat: When I start talking about big ideas or math, I feel it would be disingenuous of me to not include a warning that what I say cannot be held against me. There's a kernel of truth in the idea that mathematical rigor isn't something physicists do well (that goes for philosophy for me as well). "That's a small step for a physicist, but a big leap for a mathematician" etc... Some or even most of what follows may be in fact be complete nonsense.

Just so it doesn't get lost in the shuffle below you'll likely find this paper extremely interesting and relevant. I would probably be closest to the fundamentalist position as strange as that is for me to say. I would never offer proof of this claim or that it is wholly secure from being dethroned. It remains a somewhat unsubstantiated belief arising from my own experiences, education, seeing the at times breathtaking and insane effectiveness of math in application to reality, the aesthetic quality of mathematical elegance/beauty etc... Is it just subjective delusion? I don't know (a key theme in this response).

"If the only tool you have is a hammer, then everything looks like a nail"

But if everything we can meaningfully say about a reality relates to the nail, in what sense is it independent of the hammer? I'm stretching the metaphor here but I think you get my point. If math provides the far and away best description of reality & something like a perfect theory exists somewhere (maybe not even accessible to humans but inductively deduced so as to be written in the language of math), would it be possible to speak only in math and reject all other talk as superfluous?

But I don't believe that the formulas we use currently are the only means through which we can reach mathematical "truths".

What else? What other modes of thought or approaches could come as close or do better to help us reach "truths"? If all encountered phenomenon is necessarily subjective, including perception, is it really meaningful to discriminate between so-called objective reality and our subjective experience of it? If math remains supreme in the subjective realm, does it not make sense to also consider it supreme in the objective sense?

Maybe a good test of this idea would be if we were to encounter some phenomenon completely resistant to mathematical modelling. But then how would you definitively know that math is disjoint from the phenomenon? We invent (discover?) new math all the time. When would it be appropriate to say that existing and all possible new areas of math remain incompatible with this reality? The last thing I would want is to degenerate in reasoning and say 'I believe because I believe' but in some sense I wonder if these are unanswerable questions. It certainly would be interesting to see how other forms of life model reality provided we (and they) survive long enough to encounter them.

It seems to me that the universe is wholly deterministic, we just have yet to determine the determinate of these seemingly probabilistic functions.

A longstanding debate in QM was whether it was really a complete description of reality with it's probabilistic framework or whether there exists some underlying determinism that lies undiscovered. You're likely familiar with Einstein's somewhat misleading quote "God does not play dice." Well QM has advanced somewhat since his death and it's not looking too good for his side. Bell's inequalities contain one of the most disturbing results in the whole of the scientific literature. QM remains mysterious and counterintuitive.

If all of higher-physics rests on the foundation of the Theory of Relativity and (hypothetically) it is found to be flawed; yet, we still get seemingly valid results with inherently flawed formulas, what does that mean? (I'd honestly like to know)

This is an interesting point. Consider that Newton's laws still work very well in most everyday cases. Of course nowadays we know that Newton's laws are metaphysically untrue (whatever that means) but it doesn't change the fact that they were successfully implemented for over 2 centuries to explain phenomenon with my favorite example being the prediction of the existence of the planet Neptune. We had to go to the limits of Newton's gravitation to find fraying edges that required further explanation. If some fundamental theory of physics is overturned (like relativity) it would simply mean there exists some further extremum where it doesn't hold. Actually much of what's done in physics is approximation; there's very few experimental systems that are exactly solvable so we have to work in some approximate framework where we try to refine theory more and more by pushing experiments further. With that said are the current theories of physics "metaphysically true"? I don't know. I doubt it and a very open question that's been echoed by other scientists is whether we can ever really arrive at such theories.

If, in your original comment, you had changed "math" to "logic", then I would have agreed with you immediately.

There is a philosophy of mathematics called Logicism that actually believes math is simply logic. I don't really know enough about these philosophies or math in general to have an opinion on the school I'd subscribe to.

But I don't believe that the formulas we use currently are the only means through which we can reach mathematical "truths".

This may have been a deeper point than you were intending but Godelian incompleteness (that any formal system as strong as arithmetic can't be both consistent and complete) may have some implications about the limits of our knowledge. That there exist mathematically true but unprovable propositions about such a formal system. I know some mathematicians who say that incompleteness isn't as profound as I make it out to be but I don't know. I would definitely need to read more on the topic before being able to even come close to articulating a coherent answer either way.

Also also, consider the [3] Double-Slit Experiment. The results hint at the possibility that we, as observers, play some role in the structuring of reality. Or in the very least, that our discoveries about reality are relative only to the specific "tools" which we used

Again some of this relates to the fact that QM is very poorly understood
in terms of physical processes: what is an observer? what really is the wavefunction? does it make sense to talk about collapse of the wavefunction? Part of my own naive and at the time (maybe still) uneducated objection when I first started studying QM was that these concepts weren't rigorously defined. Of course there are some today who think we need a better understanding of QM fundamentally before we can formulate more advanced theories but again that's by no means accepted by all scientists who may consider such questions more philosophical than scientific. Just as an aside, you may enjoy reading about the delayed choice quantum eraser experiment as it's the source of a few good 'wtfs' concerning QM.

Also, consider that the results from CERN have been [2] replicated.

That's true but only replicated on the same instrumentation. I know some of this is written in the article but just to reiterate, there was some question as to the statistical method used by the neutrino team in timing events. When you have such a long pulse time, it becomes just about impossible to say when an individual neutrino (given how weakly they interact) leaves CERN and arrives at Gran Sasso. As a result the team uses statistics to arrive at the time difference. All this repeated experiment did (using a much shorter pulse length tying each neutrino detected to a neutrino leaving) is rule out that statistical method being a source of error. There are still a bunch of potential sources for error in the experiment (e.g. problems with GPS measurement, slight change of the index of refraction of the photomultipliers). It's the same sort of idea behind trying to accurately measure a table. If you use a ruler that's 13 inches long instead of 12 inches to measure your table, your answer may hover consistently around some number but that number may in fact be off from the true value. These sorts of experiments require a lot of things to work accurately in conjunction and it isn't necessarily surprising that there's some niggling problem that hasn't been found. If Fermilab or another facility reproduces the results that would be very interesting (very doubtful IMO but I'm watching with interest). I'm actually hoping for such a result despite it's relatively low probability. It would really throw a good & interesting curve ball into theory even if it doesn't require a scrapping of SR.

I would very much like to continue my discussion with you, even beyond this topic. You seem like a very knowledgeable dude/dudette and I would love to pick your brain.

Let me return the compliment and say that I too would be interested in talking more. You seem to have a great aptitude to picking out relevant arguments about some very weighty topics. You're wasted in psychology; come join us on the dark side of the hard sciences :P.