Fluency only comes with time.
How many words did you have to say before you were proficiently fluent in your first language? My guess is that it was millions, starting with the babbling in your first year, through years of talking as you acquired more sophisticated language skills.
(Or put another way, having finished 2 Duolingo courses and spent many hours studying both Italian and Swedish, why I am I still so far from fluency?)
There is a long time between competent and fluent, and during that time, you need to spend hours upon hours clarifying your understanding of language.
Of course, this is true of musical language, too. The fluent musicians I know have spent years playing after they learned to play their instrument. They have fine-tuned their understanding of their instrument, built ever-deeper instincts, and increased the nuance with which they interpret and play music.