My PS4 doesn't like going from the TV screen to a smaller monitor?

So I have a monitor in my room, and a TV elsewhere.

I decided to plug it up to the TV for a bit and it worked fine -- however it just gives me a blank screen when I go back to the monitor. When I plug it into the TV again it works.

It's not an HDMI port being bad I think -- I use the port on my PC for my computer and it is still working fine.

I suspect it has something to do with the size differences. They are the same resolution but the TV is larger.

Does anyone else have this problem? If so, how can I fix it? I really want to play bloodborne so I can get angry in my own room.

Thanks in advance!