This is exactly the problem of taking Japanese classes: you study and cram in order to pass exams and not fail, not in order to truly learn.
Also, while there are methods that help to retain and get used to the kanji (like Remembering The Kanji), only doing stuff in the language and eventually using each kanji will ensure that one truly becomes “one with the kanji”!
]]>