Hamlet is exploring alternative ways of processing reality and reaching his goals, which are not necessarily mad, but which appear mad. The method to his "madness" may be unethical but that is a different discussion. He creates chaos and tension to establish desired outcomes by allowing for arational insights and impulses to feed his narrative and more premeditated structural vision.
Hamlet is far too on top of things to be mad. Hamlet’s intellectual brilliance is first brought out in Act I, scene V when he plans on acting mad to confuse his enemies. His insults at Claudius and Polonius in Act II and Act V Sc. III are rooted from deep intellectual thought rather than erraticism, showing that the feigned madness runs through the play's entirety.