Decoding Neural Networks: An Information-Theoretic Guide to Interpretability, Error Analysis and Efficiency