Is there anything i can do about horribly slow/inconsistent download speeds?

Sometimes it's decent, other times it is stupidly slow. Like 1-2 MBps. I have a 100/40 connection. I'm not expecting full speed of course, but this terrible speed is just irritating. I have tried using a proxy, wired, got a new router because i needed one (did not fix it)