Yes! You should stop eating them immediately!
Seriously, there is almost no chance of a link between lithium-ion batteries and cancer. Some may be nickel or chrome plated and some people may be sensitive to handling them, but its more an allergy thing than cancer. In actual use, they will be inside something and the only time you might contact one is when you replace it.
They emit no radiation and no fumes. As long as the case is intact, they are perfectly safe. If they get physically damaged enough to leak, the greatest problem is that lithium is a highly reactive metal that does not play nice with water. There is a potential fire hazard and the resulting lithium compound created is caustic.