Generally you have to report it and sometimes the government will extend your EI if the official believes you need more time to improve your French by attending some language programs.
According to my understanding, EI is a kind of allowance to support your job hunting: you have to be a full time hunter to be eligible. After 8 hours day time, I think you are free to take French or other courses.
Anyway, talk to the government in person and you won't lose anything.
Several years ago, they deferred my EI because I took a part time French class. I visited CLE, and the lady in charge of my case suggested me to take some more formal courses and she also mentioned the government will pay the tuition and EI allowance throughout the program.