It can be configured to search hostname too:
https://winscp.net/eng/docs/ui_login#incremental

Thanks for the quick response!
Unfortunately Ctrl+F only searches through the labels, not through the Host Names, which is what we specifically need.

You can do Ctrl+F and type the IP address. You cannot paste it.

I'll look into improving this:
Issue 2415 – Allow pasting the search phase during incremental search

I have approximately 850 hosts in my sites tab. Sometimes I know the IP I want to connect to, but I forget where exactly it was placed in my sites tab. So I want to be able to just do Ctrl+F, paste/type the IP and find the specific row.

Does this feature already exist? If not, could it be an idea to implement such feature?